{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本书配套视频课程：[解剖深度学习原理，从0实现深度学习库](https://ke.qq.com/course/2900371?tuin=ac5537fd) \n",
    "\n",
    "更多代码或学习资料将向购买视频课程或书的学生提供。\n",
    "\n",
    "\n",
    "+ 博客网站：[https://hwdong-net.github.io](https://hwdong-net.github.io)\n",
    "+ youtube频道: [hwdong](http://www.youtube.com/c/hwdong)\n",
    "+ bilibili网站：[hw-dong](https://space.bilibili.com/281453312)\n",
    "\n",
    "## 3.5 逻辑回归\n",
    "\n",
    "\n",
    "二分类交叉熵损失：\n",
    "\n",
    "$$\\mathcal L(\\pmb w) = -\\frac{1}{m}\\sum_{i=1}^{m}\\bigl (y^i log(f_{\\pmb w}({\\pmb x}^i)) +(1-y^i) log(1-f_{\\pmb w}({\\pmb x}^i))\\bigr ) $$\n",
    "\n",
    "将样本写成行向量形式，梯度$\\nabla_{\\pmb w} { L(\\pmb w)}$也写成行向量形式，可表示为：\n",
    "\n",
    " $$\\nabla_{\\pmb w} { L(\\pmb w)} = \\frac{1}{m} {\\bigl( \\pmb f- \\pmb y \\bigr )}^T X =  \\frac{1}{m} {\\bigl( \\sigma(\\pmb X \\pmb w)- \\pmb  y \\bigr )}^T X \\tag{3-38}$$\n",
    " \n",
    "将样本写成列向量形式， 将梯度写成行向量形式，则：\n",
    "\n",
    "$$\\begin{aligned} \\nabla_{\\pmb w} { L(\\pmb w)} &= \\begin{bmatrix}\n",
    "\\frac {\\partial L(\\pmb w)} {\\partial {w_0} }  &\n",
    "\\frac {\\partial L(\\pmb w)} {\\partial {w_1} } &\n",
    "\\frac {\\partial L(\\pmb w)} {\\partial {w_2} } &\n",
    "\\cdots &\n",
    "\\frac {\\partial L(\\pmb w)} {\\partial {w_K} } &\n",
    "\\end{bmatrix}= \\frac{1}{m} {\\bigl (\\pmb f-\\pmb y\\bigr )} {\\pmb X}^T \\end{aligned} \\tag{3-39}$$\n",
    "\n",
    "给逻辑回归的损失函数添加正则化，即$L(\\pmb w)$为：\n",
    "\n",
    "$$L(\\pmb w) = -\\frac{1}{m}\\sum_{i=1}^{m}(y^i log(f_{\\pmb w}({\\pmb x}^i)) +(1-y^i) log(1-f_{\\pmb w}({\\pmb x}^i))) + \\lambda \\|{\\pmb w}\\|^2 \\tag{3-40}$$\n",
    "\n",
    "相应的，$L(\\pmb w)$关于$\\pmb w$的梯度就是：\n",
    "\n",
    "$$ \\nabla_{\\pmb w} { L(\\pmb w)}  = \\frac{1}{m}\\sum_{i=1}^{m} (f_{\\pmb w}({\\pmb x}^i)-y^i){\\pmb x}^i+2\\lambda \\pmb w \\tag{3-41}$$\n",
    "\n",
    "如果每个样本$\\pmb x$是行向量形式，多个样本的$\\pmb f, \\pmb y$和模型参数$\\pmb w$都是列向量形式，则可以写成如下的向量形式：\n",
    "\n",
    " $$\\nabla_{\\pmb w} { L(\\pmb w)} = \\frac{1}{m} {\\bigl( \\pmb f- \\pmb y \\bigr )}^T X + 2\\lambda \\pmb w =  \\frac{1}{m} {\\bigl( \\sigma(\\pmb X \\pmb w)- \\pmb  y \\bigr )}^T X + 2\\lambda \\pmb w \\tag{3-42}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.2 逻辑回归的numpy实现\n",
    "\n",
    "#### 1. 生成数据\n",
    "\n",
    "用np.random.normal()生成服从不同正态分布的2组二维坐标点数据集合Xa和Xb，每个样本表示二维平面上的一个坐标点。Xa中的样本是围绕中心点(10,12)的正态分布的采样点，Xb中的样本是围绕中心点(5,6)的正态分布的采样点。代码用不同颜色表示这些样本以区别它们所属的类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[13.52810469 15.76630139]\n",
      " [ 8.20906688 11.86351679]\n",
      " [ 4.26163632  3.3869463 ]\n",
      " [ 6.04212975  4.47171215]]\n",
      "[0. 0. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Persistent random data\n",
    "np.random.seed(0)\n",
    "\n",
    "n_pts = 100\n",
    "D = 2\n",
    "\n",
    "#x0 = np.ones(n_pts)\n",
    "Xa = np.array([#x0,\n",
    "               np.random.normal(10, 2, n_pts),\n",
    "               np.random.normal(12, 2, n_pts)])\n",
    "Xb = np.array([#x0,\n",
    "               np.random.normal(5, 2, n_pts),\n",
    "               np.random.normal(6, 2, n_pts)])\n",
    "\n",
    "\n",
    "X = np.append(Xa, Xb, axis=1).T\n",
    "#y = np.matrix(np.append(np.zeros(n_pts), np.ones(n_pts))).T\n",
    "y = (np.append(np.zeros(n_pts), np.ones(n_pts))).T\n",
    "print(X[::50])\n",
    "print(y[::50])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEYCAYAAACHjumMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2df5gdZZXnP6d/2R2BAB1AIOkEBIMjAdSo6IjDGkGDIq6Ps4oRI7ibAZd1wuoImt0hzEwecXQERhddFFTsCMNo+KWEQcPMiOyAdlDSQnQQCUkjQghDB0yH7k7O/lF1O7dvV9Wt3z/uPZ/nuU/frqpbdaq63+8973nPe15RVQzDMLKgo2gDDMNoXUxgDMPIDBMYwzAywwTGMIzMMIExDCMzTGAMw8gMExgjMiKyWkQGi7bDKD8mMBVCRN4sIv9PREZF5FkRuVdEXle0XVEQkS0iMiYiz4vIc+79nC8iof4XRWSBiKiIdGVsZy7XaXVMYCqCiBwAfB/4EnAwcCRwGfBikXbF5ExV3R+YD1wOXAxcW6xJRhaYwFSHVwCo6g2qukdVx1T1LlXdBCAiLxeRu0Vkh4g8IyJrReTA2oddz+EvRGSTiPxBRK4VkcNEZL3rTfxIRA5yj619e68Qkd+JyJMi8gk/w0TkZNcTeU5EHhSRU8PckKqOquptwPuB5SJyvHu+d4rIz0Vkp4hsE5HVdR/7sfvzORF5QUTeGOLeLxaRJ9z7/LWILHG3d4jIJSLyqPvZm0TkYL/rhLknowFVtVcFXsABwA7gW8BS4KCG/ccApwEvAQ7BaSBX1u3fAtwHHIbj/TwNPAC82v3M3cCl7rELAAVuAF4KLAK2A29z968GBt33R7p2nYHzhXWa+/shPvexpXaehu1bgQvc96e61+wATgCeAt7TYFtXmHsHFgLbgCPqPv9y9/1K95nMdT/7f4Eb/K5jr+gv82AqgqruBN6M80//NWC7iNwmIoe5+3+jqj9U1RdVdTvwReBPGk7zJVV9SlWfAO4B7lfVn6vqi8DNOGJTz2Wq+gdVHQa+AZztYdqHgDtU9Q5V3auqPwSGcAQnCr/D6fqhqv+iqsPu+TbhCF3jvUzR5N734IjHH4lIt6puUdVH3X1/BqxS1RH3GawG3mdxl/QwgakQqrpZVT+iqnOB44EjgCsBRORQEbnR7QrsBAaBOQ2neKru/ZjH7/s1HL+t7v3j7vUamQ/8qds9ek5EnsMRwsMj3t6RwLPuvbxBRP5ZRLaLyChwvse9TBF076r6GxxPZTXwtHtc7T7mAzfX2b0ZR5AOi2i74YMJTEVR1V8B38QRGoDP4ng3J6jqATiehSS8zLy69wM4XkYj24Bvq+qBda+XqurlYS/ijoQdCfzE3fQd4DZgnqrOBr7Kvnvxmv4feO+q+h1VfTOOoCjwuTrblzbY3ut6eFZmIAVMYCqCiBwnIp8Qkbnu7/Nwuiz3uYfsD7yAE5Q8EviLFC77v0Vkloi8CjgX+AePYwaBM0Xk7SLSKSK9InJqzc4m93SAiLwLuBEnpjNcdy/PqupuEXk98MG6j20H9gJH123zvXcRWSgibxWRlwC7cTy1Pe7urwJrRGS+e+whInJWwHWMiJjAVIfngTcA94vIH3CE5ZdAbXTnMuA1wCjwA2BdCtf8V+A3wAbgC6p6V+MBqroNOAv4DE6j3IbTwIP+t24XkefdY1fhxEzOrdv/MeCv3GP+Erip7nq7gDXAvW7X5mSC7/0lOEPhzwC/Bw51bQW4CsdTusu91n04z9jvOkZERNU8QWM6IrIAeAzoVtXJYq0xqox5MIZhZIYJjGEYmWFdJMMwMsM8GMMwMsMExjCMzKhUSvScOXN0wYIFRZthGG3Hxo0bn1HVQ6J+rlICs2DBAoaGhoo2wzDaDhF5PM7nrItkGEZmmMAYhpEZJjCGYWSGCYxhGJlRqSCvYQQxPjzM7g0b0NFRZPZsepcsoWfRoqLNamtMYIyWYHx4mLHbb4eJCQB0dNT5HUxkCsS6SEZLsHvDhilxmWJiwtluFIYJjNES6OhopO1GPpjAGC2BzJ4dabuRDyYwRkvQu2QJdHdP39jd7Ww3CsOCvEZLUAvkJh1FspGodDGBMVqGnkWLEomBjUSlj3WRDMPFRqLSxwTGMFxsJCp9MhcYEblORJ4WkV82bP8f7kLkD4nI32Zth2E0w0ai0iePGMw3gS8D19c2iMh/wllL5wRVfVFEDs3BDqNCFBFs7V2yZFoMBrCRqIRkLjCq+mN3nZ16LgAudxccR1WfztoOozqkFWwNEim/fZNbtzKxcSOoggjdJ55oAd4EFDWK9ArgFBFZg7Oc5ydV9WdeB4rICmAFwMDAQH4WGoURFGwN29iDRArw3De5dSsTDz7oiAuAKhNDQ+wCZr3znYnvKwvKPqxelMB0AQcBJwOvA24SkaPVYw0VVb0GuAZg8eLFtsZKG5BGsNVXpNavR3fv3icidfumPJcGJoaGGB8YKFXDhWoMqxc1ijQCrFOHn+IsMj6nIFuMkpFGsNVXpMbGPEXE2en//VXGoeoqDKsXJTC3AG8FEJFXAD04i5MbRipp/7FGfkR8d5VxqLoKw+qZd5FE5AbgVGCOiIwAlwLXAde5Q9fjwHKv7pHRnqSR9u85IhREdzfdJ57IhM+qFTJ7duniHTJ7tqeYlGlYPY9RpLN9dn0o62sb1SVp2r+XSOn4OIyNzTxYhL4zz6Rn0SJ2wUyR6e6m69hjSxfvqMKwus1FMlqWRpFqDIoC0N09JS7gjBaNDwzM8FTSGNlKm7QmeGaJCYzRNoRtkF7e09i6dZ7nLDrekdTTyxoTGKOtiNsg04x3lC2WkyU22dEwQpBWQataN60mVrVYzvjwcFqmlgoTGMMIQc+iRfSdeSbS17dvY1f0DkAVclfSxATGMCKgk5P7fhkbi+x9VCF3JU1MYAwjJGl4H+1WEsKCvEblKCpImob3UYXclTQxgTEqRaET/Pr6vBP16uMyTahC7kqamMAYlaLIhDcBvOaz+M9g8qbsuStpYjEYo1IUGSRVL+8lYLthHoxRchrjLX7dlDyCpFWYXFg2TGCM0uIVb6GzEzo6YO/efQemFCRtFjxutwBtGpjAGKXFM96yZ4+T7NbTk2qQNEzwOEqAtp2mAwRhAmOUlqCqdLM/9alUrxU2eBwmQFuFUpZ5Udi6SO6+T4qIioiVyzRmkGdSWprB43abDhBEIesiAYjIPOA0YGsONhgVJM+YR5oB3LJPB8iz+1bUukgAVwCfAm7N2gajmjTGPBCBiQnG1q1j9/r19C5dmlr8Iy0xGx8eduz0qAAb1/NKUxDy7r4VEoMRkXcDT6jqgxJQaNk91tZFamNq//Rjt9wybeRIx8YYu/XWacdA/AaURobt1LW9ykvH9Lx2/eAH00p4JhWEvBMVcxcYEZkFrAJOD3O8rYtk7N6wYfqwdI09e2Y0DL8GNLZ+fdMGlDTD1vPaMK3mbz3NPJPx4WHvIuQJBCHv7lsRHszLgaOAmvcyF3hARF6vqr8vwB6j5AT98zfu8z12bIzx4eFEXYtm3o3vtVU9xaWZpxUUFK6/VpQuVN7JgrkLjKoOA1OL3YvIFmCxqtq6SIYnfo2iti/ssX7f+mE8iTDdriiNN0xXJUhYa+eM2iXMO1kwj2HqG4B/AxaKyIiIfDTraxqtRe+SJU72biOdnTMaRlBD8WqwYUpYhh12jlJWM0xXJcirqJ0z6pD4VGU+99wye7Zn9y0tilwXqbZ/QdY2GNVmKtC7fv2+eUjd3UhXlzOitGHDlNfRs2iRs/50yPlKvjGbm29mbN26QI+ocXuUQHEYb8d38bieHl8bmm2v2ZlXwp9l8hqVoL5RTHkdrog0dgt6ly4N3Q0IipsE7sdbsMI23jBdFU9hBRgfn7pfXwEUYfSyywqfpmDlGozK0axbEKUbEDu4mTBuEdZGX2Fw79ezWwbTBLLIVQvMgzEqR5huQSJPIgTdJ56Y2Cto9Mp2b9gw1S2reR3jw8PeVfRw7tczGbExD6fAFShNYIzKkeZQa6gG6sHE0BDjAwOpNNqgkaCgoera/dYL1ehll3keW9Q0BRMYo3KkPdTqFd8J49GM3XLL1OeTENTlCxIGr/stW1EsExgjN9KaU5Nl4exIHs3eval0PYK6fL6C0ddXijyXZpjAGLmQ9iS7ZjGWJGI2w6PxWfge0ul6BHkdXcceO3O6QHc3vUuXep6rbKsWmMC0GUVVWstzkl2aYhaUVwPpdD38vI6uY49l4sEHZxzfLMBcplULTGDaiCIrreU5yS6OmNULb61bVBPg3qVLnZnbe/ZM/1BHRypdDz+vw2/y5OQjjyS+Zl5YHkwbUWSltTJXp2ucLtCYQwLQd9ZZ0xe+7+uj7z3vCVU+c+eVVzJ62WXsvPLKtlvH2jyYNqLIf9gyV6fzLbMAUwJ8wMqVkb28sB6j73EFLtGSFubBtBFFLrye5yS7KJMOobnAxhXgsB6j33ECke6jjJgH00Zk7UU0CyDnFXyMOpISNKGxtj8OYT3GoNUT+t773tKMCMXBBKaNyHIIs2xLdUQRs8DpAgkEOGxXLei4Mo0IxcEEps3I6h92bP36whalT4pfcl1SAQ7rMZYtOS5NTGCM0Ph1gZpNyCsTfveQhfCG9RjLlhyXJpkLjIhcB7wLeFpVj3e3fR44ExgHHgXOVdXnsrbFiE/SCXllIOgeILupB2HOU/WukB+iIWaOJrqAyFuAF4Dr6wTmdOBuVZ0Ukc8BqOrFzc61ePFiHfKqsm5kzs4rr/SNEwR5KX3vfW9pGo7vPfT1oZOT07soHR3IS16Cjo21lEcRFxHZqKqLo36ukIXXVPWuul/vA96XtR1GMtKckBeVtKY3BI3WzGDvXt+KeUZ4ypAHcx6w3m+niKwQkSERGdq+fXuOZhn1BOXQ+Oad+EzIi0KYotxhSdRda9O1pZNSaJBXRFYBk8Bav2Ns4bVyEDTSESZIGdcLSXOSpOfM5AiECVgXNZm0rBQmMCKyHCf4u0SzDgQZiWkmIkFByiQ5MmlNbxgfHvacmRyFZh5Q3PtsZVEqam3qdwAXA3+iqruKsMGITtyRjiReSFoV2gLnG3nR2Tl99nSIvJS4s7jLlKCYNnkMU98AnArMEZER4FLg08BLgB+6y8fep6rnZ22LUQxJvJC0ktBCezzd3fSdeSbg7635eRxx7jPvxejzpqiF167N+rpGeUjihaSVhOY7nN7Xh/T0+Hb7GgnyOOLcZyuUZAjCMnmNzGnmheQxSdLPhr6lSyOdO8jjiONtla1Id9qYwBiZE+SF5BWDSMsTCvI4mt2n1/ZWnocEJjBGTvh5IVnGIMaHh6cvu9rT4wRvfY5NY01pr/sMI6I2imQYGZAkBhEkCuPDw866RXv31n1gfNr56+chhfWi4ngczUS0VechgQmMUTBxYxDNvILdGzZMFxcv6rNzQ3pRcTyOVg/kBmECYxRK3BhEM68gbOMNOs5vX1SPo9UDuUGUYS6S0cbErdXbzCsI23hl9uzMaxVHrRHcSpgHYxRK3DT5Zl5B75IljN18c/BC9nWNPKwX1Rg4lr4+epsMdbd6IDcIExijMKIs69HYOH3r6I6P75tp7bWudHe3U7Hfo5E3EwCvwLGOjTmLshE8rF62QG5e858yLziVJlZwKnvWroVVq2DrVhgYgDVrYNmybK7lVwCqvibu1PKpjQlybjr/tGHoENSvmhi1cfna6573gJUrQ9tRJI3CDkw9U9+VF8pacMqoDmvXwooVsMudfvr4487vkI3I+AZY61ZW9CyvULcY2u4NG3zXjfa7ZjOvKepco8B7KSF5zn+yIK8xxapV+8Slxq5dzvYsSBJErS9AFe2iErgYWlCBqyB7qzQilOewuQmMMcXWrdG2J6V3yRLfzNqmuOtER2rY3d2+Qd9a42o214gOjybT2TktIJxkPeo8yHOFTxMYY4qBgWjbU6G+5koExP3pOQQ87UDnyNrwd7PG1WyuUd973jMlbuCMIvWdddb0DOKUSnxmRZ7D5haDMaZYs2Z6DAZg1ixne9pMBRpjUou7TA0Br18/MxbjE7gMGpKOM9eonirUd8lz2LyodZEOBv4BWABsAf6Lqv5H1rYYwdQCuXmMIoWpMCezZ6Pj456jRPWeSK3Rhxl6bda4ks5ursq0gLyGzfPwYL4JfBm4vm7bJcAGVb1cRC5xf2+6LpKRPcuWJReUMA09sMHVeR5+Q6peDT6NRc6aCVCze2vnaQFeFLIuEnAWThlNgG8B/4IJTEsQNnnOt8KcyLRuTRFZsH4CFObeWr2+S1SKisEcpqpPAqjqkyJyaEF2GCkTNgbhW2HOI2aS1J1PmrVa//kZ1A1xT12jrw/t6oIQq0K28ooCUIEgr4isAFYADGQ6nGGkQdgYRFqeSbMGmrRinmcXzePepl1jbMwRyybL5rb6igJQnMA8JSKHu97L4cDTfgfawmvVIkoMIg3PJEyluCSjOqGWOwlI3guaU6Xj46mOOJXRGyoqD+Y2YLn7fjlwa0F2GCmTZ45FkHjUiDuqU0uWazr6EyJ5r3a+xvwYvzlUcUacypp/k7nAuOsi/RuwUERGROSjwOXAaSLyCHCa+7vRAsSt7xKHMOIRJ2u1sbH6ETZ5D6It/BZnxCmM2BZBUesiAbRnWN1IjaDuWGBgNk4N3YbPN4pms5GjKAu/xfH2ypp/Y1MFWpi1a2HBAmf6zIIFzu9p87GPQVeXE4bo6oI/e/8zubnqft2xrmOP9fVAwnhUQY1S+vqgq4uxdeum5hmF8tpEfE4oqXh7ec4vikLpR5GMeORReuFjH4OvfGXf73v2wDU39bPnt0v4u3fduW9HRqnyfiNRfh5I2Jotvp5RXx86OekbVA4aivatrKeaSh2ZsubfmAfTouRReuGaa7y2Ct/cOLMuUVaues+iRRywciWzL72UA1aujL1GdD1+npFC5DhHszlXaXkYeca+omAeTIuSR+kFv4nQe3Tm91aernrSdH0/z2hs3TrP4yMvbl8jZQ+jbGU5wQSmZRkYcLpFXtvTorPTW2Q6ZWYd3CgNKWk+RxrdBa/G6hc0jrO4PVAKDyNrrIvUQtQHdV94YaaXn3bphVpMp5GP/umzsV31NPI5suou+NWe6Tr2WN/PBAVfW11cwDyYlqExqLtjh7MUc38/PPtsNqUXrr7a+XnNNY4n09np2HD11XOAeIHLtOqpZNFd6Fm0iMmtW2fUCZ548EHGBwY8r5d38LVs2byhPRgROU1EviYiJ7m/+3x/GUXgFdQdH4f99nNW2diyxV9ckgxnX301TE46gySTk/tEJy5lzeeoMfnIIzM3BgR68wy+ljGbN4oH8zHgXOB/uQWjTsrGJCMOcYO6ea8k0Iyy11OJI4B5BV/LWE0vSgxmu6o+p6qfBE4HXpeRTUYM4tbTzXslgWaUfZnVsia0QTm9vygC84PaG1W9hOkV6oyCWbPGCeLWEyaom/dKAs0oaz5HjTILYBnFr2kXSUSuBC5S1WkznlX1S5lZZUQmbj3dPIazo1LGfI4aZV5nuozZvGFiMC8At4nI+1V1l4icDlyqqn+csW1GROLU081zJYFWoawCWErxU9WmL+CDwM+AnwD/BJwS5nNpv1772teqkT6Dg6rz56uKOD8HB9M93qg+wJDGaLNNYzAisgT4b8AfgEOAj6vqPZkpnhFIFjOkly1zhrHrh7P9rlMbdXr8cWdoujbqlMVMbaMFaKZAwN3Am933i4BfAG+No2ZJX+3uwQwOqs6apeo0bec1a1b6HkTQdebPn7699po/P10bjHJBTA9G1G8auQ9uDd3vqeqbkoqbiFwE/FdAgWHgXFXd7Xf84sWLdaghi7KdWLDAOyA7f77jeeRxna1bvSsPiDgekNGaiMhGVZ05Tb4JkeciqbPcSOKwtIgcCXwcWKzOio+dwAeSnreVyWtIOeg6haxfbVSWWJMdVdW7WnF0uoA+EekCZgG/S+m8LYlfIz74YO/tceM1QSISN9/GaFPi9KvSegF/jjMMvh1Y63PMCmAIGBoYGEivU1lBBgdVu7tnxj96embGYZLEa5p9tuyjSGW3r4oQMwZTpLgchBNAPgToBm4BPhT0mXYP8qqq9veHC7KGDcb6NcbBwenX6u8vd0OtD0CLxBNWw5+4AlNkPZi3AY+p6nZVnQDWAYkDx63Os896b2+Mm4SJ1zQbcq5ftmfHjvIOR9ffB8wMQhc5t6rdKVJgtgIni8gsERGcwPHmAu1JnSxyVvziIx0d088fJhgbNNExjUmQtcXLRi+7bKoCfxZ42dpIUXOr2p3CBEZV7we+CzyAM0TdgbtEbCuQVUKaV5AVnIJP9ecPE4wN8nKSjljlWZskjE02ylUMhZbMVNVLVfU4VT1eVc9R1ReLtCdNmnkAcb2bZcucCnKdnTP31Z+/dtz8+U6Oyvz5zu/1c5WCRqU6fP4z6j8TdA95rjTYTDxslKs4rCZvRgR5AEm9m2XL/JPa6q/rNQWgHi8vp6cHdu70Lubd0+PU+u3ogDlz4Nxz/e8hz9okXvdRW+fMS1iN/DCByYigGEga8Y00Et68vJz99/deZaOjwxGSHTv2/Ww8btcuWL7cOXbRVRfxj5uOn3GeLGqTeN3Ht7/t2BlUKtTIHhOYjAiKgUSJb3h1Q9audTyJRuJ0BRq9HL9Rqr17w63dvmeP07C3/ccBfPz2M6eLTIa1SZp5a0ZBxBnbLupVtTwYvxyTKDkqjQlv3d1OYl3jZ9PKU/GzLe5r3kGj+tzq1Tp6xRX64qZNyQ00CoGqJdrFeVVNYPwIm2UbpbHXBCxpBqufbX4Jfs1eIsmfl1E8JjApk3W6eZjzN2akNnulVcrByzYv4QkrfEb1MYFJkbzqrjQjigfT2Zl9Aw9Kx+/pmTlPqvbMbG5Q9YkrMBbk9SCNUZ40sni9AsXd3c5wcT2zZvkvRN8YOE5iVy2QquqM0tSP2lx3HXzjGzPzbsAq4LU1cVSpqFdeHoxf1yRsPCFND8ivu9K4LUzguAjPzCrgtQZYFyk9kjaKvBtV48xnP/GIY1fS7k1SsTbKgQlMiiT9ps+zUfkFX72GraPalYbHYx5Ma2ACkzJJvrnzbFRRrhXVrjTuoywBcyMZJjAlIs9GFcUriVqpzuu8cTyx+hhRbbSrsbiVjTKVGxOYkpFXo/ETgv7+aHZ5iY+feMXxxPzE7YILzMOpAiYwbcrgoPfUge7udLp1aZWf9Dt/Hvk7RnLiCkyheTAicqCIfFdEfiUim0XkjUXaUwUa81jAmQHdyMREtLwdvwmYqtDf719TJun5w+bvGNWk6ES7q4A7VfU44ERKWjIzi9KXce1oTFo77zyndIIXURppUJmHsTEnsS7JLGW/83sVzmpmj1Eh4rg9abyAA4DHwFldMsyriC5SmUZBos509utmxJlrlLTLUnQMxgLJyaBqMRjgJOCnwDeBnwNfB14a9JkiBKZMeRxRJj/6NdILLvCPqwwO+p8vjRyeoABz1hNLy/IlUVWqKDCLgUngDe7vVwF/7XFcoQuvlSkTNYoH49V4BgebjwylJahhpzjkQZm+JKpKFQXmZcCWut9PAX4Q9Jl292DClkzws61ZTZcwC5eFEQkvO4NmW2dNmb4kqkrlBMaxmXuAhe771cDng47PW2DCzvHJ26ZaA+/vn9loRZxukNfngsSlsRHWfm+WK5NGoaysKdOXRFWpqsCc5HZ/NuEsHXtQ0PF5TxUIO8enSLxiKj09jp1hZls3a/z1ghY2ZyVKrCgPL8JiMMmppMBEfeUpMEnXdi7azsbGFFVc6sUqqkiUzYNRLf7vVHVMYFImTL+9DN+MUctqRnl1dMQTibLFYOrtMpGJR1yBKTrRrrQkXds5LlGT+rJKSJs1y39xt8bjGpdK8VqnqFbxrr9/33F9fenaHERWS/kaTYijSkW9io7BNH7jpj06EccjiluMO+hVizM16xZF9QKK9Pgs0JsMrIuUPs1c6qAJfGlOCGzWCOqDuGG7NWG6PH7D2n4ztbO6vzSwoepkmMAUQJD3UBsujtLvT9IIgmzp6JgZrA0K3tauNzg4M24SdZZ2WveXFPNgkmECUxCDg/7Dt7UGGbZLkKQRNBu56e+fKXR+HkpjofAggYwioEU28jIE5KuMCUyBRB3JCZqEGLcRNLMhToW7ZkT9fNGN3EaR4mMCUyBRk9iCugRxG0EzG6LMrE5631muUmAUQ1yBEeez1WDx4sU6NDRUtBkzWLsWzjnHaV5hmD/fqa2Stg0rVswcNgdnKDluoaggRPy3hxniNqqDiGxU1cVRP2d5MCmwbBmcf/7MBtfT46zEWI9X3kg9YfJgvI6pzz2pp7MTli933sctmuV1vaDPH3xw+HMbLU4ct6eoV1m7SDWSligIE6Pwy5KtzT3q7/ceMYqbRetnU9DM7KBhbOsiVROsi1R9FixwMkwbqe9S+R0ThzBdtTjX8+sieXXjsuq+GekSt4tkAlMiOjq84zj1DdbvmDiEiZXEuZ6fcIURUKOcWAymBQgz/ynNuUdhzhX1ej09/jEmvyLktoJA62ICUyLWrHG6DPU0BoW9jglD1GBznOv19zuTGv26O2EE1Ggx4gRuinqVPcibBmFLUgZVtfN6JQmuehW1ipOJW3SinREfqppoB3TirCrw/WbHNhOYqo5QJLW7WcW6pKn4QeeOKhBV/Ru1O1UWmP8JfCepwFT12zFNu7N6BkHeS5bP18SoPFRSYIC5wAbgrUkFpqqzZdO2O4tGWcSzreoXRqtSVYH5LvBa4FQ/gSHkukhVrfdRtN1xlyHxauxpiltVvzBalcoJDPAu4Gr3va/A1L+q4MFEbWRVKWEQpmxDmh5H0cJrTKeKAvNZYATYAvwe2AUMBn2m7DGYtEpeRi2ZkPVs6DDXSFsoy/KFYThUTmCmGZGCB6NafFAwacnLqHb7idMFF4Q7X5orJ5ShPrGRHSYwJSBvt95P0IKWfg3z+XpBDCuaWXgcRX9hGPuotMCEfZVdYPJ266NU0vOyIc2VE8zjaG3iCoxNFUiRMKn+cfGqyRIlxb5+vk/tXOec46xN1N+/b/2ixpnNYdP7vdZCslnSRuFeSZRX2T0Y1VrrOdUAAAwQSURBVGzc+qBYS9g1kWoeTNXq6BrlAOsitS5B6y/V/wx61QTB6ugacYgrMFYPpgKEqckya5bT3dmxY+a+/n545pngc+VdR3ftWmeJ3a1bne7WmjXl605NTEwwMjLC7t27izYlN3p7e5k7dy7dDdPv49aD6UrNMiMzBgaaV5XbtcsRmFmzZlaMu+qq5ufKs2RCY2W72jrRUC6RGRkZYf/992fBggWIX4XzFkJV2bFjByMjIxx11FGpnNOCvBUgbE2WZ5/1D7TWAruPPz6zOHlageiwrFo1c/WDXbuc7WVi9+7d9Pf3t4W4AIgI/f39qXpsJjAloNlKAo0jNJ2d3ucZGHCO3bLF6e5s2bJPXFas2Oe5qO4TmSJGe6pU2a5dxKVG2vdrAlMw9Y1fdV93wUtkasLxrW9FGw738hhU99XCjSMuYZZX8cMq27URcSLDRb1acRQpj+kFQQl5cch7ydmiePjhh4s2oRC87htLtKsmcbsLXl0hP/w8A5FonkeNpDEUS8prH0xgCiaP7sKaNd7LvKrGC6ymEUNpJpBJumCtxObNmznqqKPY6+YQ7N27l9NPP53rr78+8rnuvPNOFi5cyDHHHMPll1+etqnexHF7inq1Yhcpr+6CXxcpzkTMrOdclaULFbWL9OKmTTp6xRX63OrVOnrFFfripk2p2PGOd7xDb731VlVVvfjii/XCCy+MfI7JyUk9+uij9dFHH9UXX3xRTzjhBH3ooYc8j7UuUgsRtruQ9Bu9cc3qGnE8pSznXEF1hrHrGR8eZuz229HRUQB0dJSx229nfHg48bkvuugivvKVr/C9732Pe++9ly9+8YuRz/HTn/6UY445hqOPPpqenh4+8IEPcOuttya2rRkmMCUgTHchzEhTEGmKQtYxlCoNY9fYvWEDTExM3zgx4WxPyOmnn87IyAif/vSnuemmm2Zk2Z5yyimcdNJJM14/+tGPpo554oknmDdv3tTvc+fO5YknnkhsWzMsk7cCBH2jh23UtePSSs9ftiy7oGwZso2jUvNcwm6Pypve9CZe/epXc/jhh8/Yd8899zT9vNPLmU4eOT4mMBUgrW/0LEUhTdasmT6VAPLPNo6KzJ7tKSYye3Yq53/44Yc599xzPfedcsopPP/88zO2f+ELX+Btb3sb4Hgs27Ztm9o3MjLCEUcckYptQRQmMCIyD7geeBmwF7hGVa8K/lR7UsVv9CSk7W3lQe+SJYzdfvv0blJ3N71LlqRy/oceeojjjz/ec18YD+Z1r3sdjzzyCI899hhHHnkkN954I9/5zndSsS2IIj2YSeATqvqAiOwPbBSRH6rqwwXaVEqq+I2elKp4WzV6Fi0CnFiMjo4is2fTu2TJ1PYkbNu2jQMPPJD99tsv9jm6urr48pe/zNvf/nb27NnDeeedx6te9arEtjW9buZX8EFVnwSedN8/LyKbgSMBE5gGqviN3o70LFqUiqA0Mm/ePH77298mPs8ZZ5zBGWeckYJF4SlFDEZEFgCvBu732LcCZ/E1Blq1TxCCqn2jGwaUYJhaRPYDvgesVNWdjftV9RpVXayqiw855JD8DTQMIzaFCoyIdOOIy1pVXVekLYZhpE9hAiPOIPy1wGZVjZ6a2IbY/ByjahTpwfwxcA7wVhH5hfvKNwJVIdLI5jWMvClyFOknQHuVC0tAGtm8hpE3hQd5jXBUcX6OYZjAVAQrM2lUEROYipB1iQSjnKRZcOq8887j0EMP9Z1ykAUmMBXBykyWnyxG+V75yldy3HHH8f3vfx+Az3zmMyxcuJAPf/jDkc/1kY98hDvvvDO5UREoRSavEQ7L5i0vWS4md9FFF3HFFVcwMTHBvffey9133x3rPG95y1vYsmVLMmMiYh6MYaRAllX40ig4VRTmwRhGCmQ9ype04FRRmAdTEJaV21pkPcr38MMPc9JJJ3nuMw/GmEZVFn83wpN1zZ6kBaeKwjyYAqhi1XwjmCxH+dIoOAVw9tln88Y3vpFf//rXzJ07l2uvvTa5cU0Qr2LAZWXx4sU6NDRUtBmJ6ehw5hM1IuKsLGCUg82bN/PKV76yaDNyx+u+RWSjqi6Oei7zYArAsnKNdsEEpgAsK9doF0xgCsCycqtDlUIIaZD2/dooUkFYVm756e3tZceOHfT39+eySFnRqCo7duygt7c3tXMWKjAi8g7gKqAT+LqqXl6kPYZRz9y5cxkZGWH79u1Fm5Ibvb29zJ07N7XzFbnwWifwf4DTgBHgZyJym62LZJSF7u5ujjrqqKLNqDRFxmBeD/xGVX+rquPAjcBZBdpjGEbKFCkwRwLb6n4fcbdNQ0RWiMiQiAy1k6tqGK1AkQLjFTWbEcK2dZEMo7oUGeQdAebV/T4X+F3QBzZu3PiMiHgsAx+JOcAzCc+RBmWwoww2gNlRNhtgph3z45yksKkCItIF/DuwBHgC+BnwQVV9KOPrDsVJeW5FO8pgg9lRPhvStKPIZUsmReRC4J9whqmvy1pcDMPIl0LzYFT1DuCOIm0wDCM72nGqwDVFG+BSBjvKYAOYHfWUwQZIyY5KlWswDKNatKMHYxhGTpjAGIaRGS0rMCLyDhH5tYj8RkQu8dgvIvL37v5NIvKaDGyYJyL/LCKbReQhEflzj2NOFZFREfmF+/rLDOzYIiLD7vlnlATM6VksrLvHX4jIThFZ2XBMJs9CRK4TkadF5Jd12w4WkR+KyCPuz4N8Phv4f5TQhs+LyK/cZ36ziBzo89nAv18KdqwWkSfqnvsZPp+N/ixUteVeOMPejwJHAz3Ag8AfNRxzBrAeJ6P4ZOD+DOw4HHiN+35/nLyfRjtOBb6f8fPYAswJ2J/5s/D4+/wemJ/HswDeArwG+GXdtr8FLnHfXwJ8Ls7/UUIbTge63Pef87IhzN8vBTtWA58M8TeL/Cxa1YMJM5HyLOB6dbgPOFBEZi46kwBVfVJVH3DfPw9sxmO+VQnI/Fk0sAR4VFWTZmWHQlV/DDzbsPks4Fvu+28B7/H4aGoTcr1sUNW7VHXS/fU+nGz2TPF5FmGI9SxaVWDCTKQMNdkyLURkAfBq4H6P3W8UkQdFZL2IvCqDyytwl4hsFJEVHvtzfRbAB4AbfPZl/SxqHKaqT4LzRQAc6nFMns/lPBwv0otmf780uNDtql3n012M9SxaVWDCTKQMNdkyDURkP+B7wEpV3dmw+wGcrsKJwJeAWzIw4Y9V9TXAUuC/i8hbGk30+ExWz6IHeDfwjx6783gWUcjluYjIKmAS8Ft+r9nfLylfAV4OnAQ8Cfydl5ke25o+i1YVmDATKSNPtoyDiHTjiMtaVV3XuF9Vd6rqC+77O4BuEZmTpg2q+jv359PAzTjubj25PAuXpcADqvqUh52ZP4s6nqp1A92fT3sck/lzEZHlwLuAZeoGOxoJ8fdLhKo+pap7VHUv8DWf88d6Fq0qMD8DjhWRo9xvzA8AtzUccxvwYXcE5WRgtOYyp4WICHAtsFlVv+hzzMvc4xCR1+P8TXakaMNLRWT/2nucwOIvGw7L/FnUcTY+3aOsn0UDtwHL3ffLgVs9jgnzfxQbcUrGXgy8W1V3+RwT5u+X1I76eNt/9jl/vGeRRmS6jC+ckZF/x4l8r3K3nQ+c774XnJKdjwLDwOIMbHgzjhu5CfiF+zqjwY4LgYdwovL3AW9K2Yaj3XM/6F6nkGfhXmcWjmDMrtuW+bPAEbQngQmcb+KPAv3ABuAR9+fB7rFHAHcE/R+laMNvcOIatf+Nrzba4Pf3S9mOb7t/9004onF4Ws/CpgoYhpEZrdpFMgyjBJjAGIaRGSYwhmFkhgmMYRiZYQJjGEZmmMAYhpEZJjBGJohTpuI09/3fiMjfF22TkT+FFv02WppLgb8SkUNxJnm+u2B7jAKwRDsjM0TkX4H9gFNV9XkRORpYhZPJ+75irTPywLpIRiaIyCKcglsvqlMLB3VqiXy0WMuMPDGBMVLHnTy3Fqcg0R9E5O0Fm2QUhAmMkSoiMgtYB3xCVTcDf41TktFoQywGY+SGiPQDa4DTgK+r6mcLNsnIGBMYwzAyw7pIhmFkhgmMYRiZYQJjGEZmmMAYhpEZJjCGYWSGCYxhGJlhAmMYRmaYwBiGkRkmMIZhZMb/B/zJuLpo6IjcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(4,4))\n",
    "ax.scatter(X[:n_pts,0], X[:n_pts,1], color='lightcoral',\n",
    "          label='$Y = 0$')\n",
    "ax.scatter(X[n_pts:,0], X[n_pts:,1], color='blue',\n",
    "          label='$Y = 1$')\n",
    "ax.set_title('Sample Dataset')\n",
    "ax.set_xlabel('$x_1$')\n",
    "ax.set_ylabel('$x_2$')\n",
    "ax.legend(loc=4);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. 梯度下降法的代码实现\n",
    "\n",
    "逻辑回归的基于动量法的梯度下降法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(z):    \n",
    "    return 1 / (1 + np.exp(-z))\n",
    "\n",
    "def gradient_descent_logistic_reg(X, y, lambda_, alpha, num_iters,gamma = 0.8,epsilon=1e-8):\n",
    "    #cost_history = []          # create a vector to store the cost history\n",
    "    w_history = []          # 记录迭代过程中的参数\n",
    "    X = np.hstack((np.ones((X.shape[0], 1), dtype=X.dtype),X))  #添加一列特征1    \n",
    "    num_features = X.shape[1] \n",
    "    v= np.zeros_like(num_features)\n",
    "    w = np.zeros(num_features)    \n",
    "    for n in range(num_iters):\n",
    "        predictions = sigmoid(X @ w)                 #求假设函数的预测值，即f(x)\n",
    "        errors = predictions - y                   # 预测值和真实值的误差\n",
    "        #gradient = X.transpose() @ errors /len(y)        #计算梯度\n",
    "        gradient = errors.transpose() @ X /len(y)        #计算梯度\n",
    "        loss_grad = errors /len(y)\n",
    "       \n",
    "       \n",
    "        gradient += 2*lambda_*w\n",
    "        if np.max(np.abs(gradient))<epsilon:\n",
    "            print(\"gradient is small enough!\")\n",
    "            print(\"iterated num is :\",n)\n",
    "            break\n",
    "        #w -= alpha * gradient        #更新模型的参数\n",
    "        v = gamma*v+alpha* gradient\n",
    "        w= w-v             \n",
    "        \n",
    "        #cost = - np.mean((np.log(predictions).T * y+np.log(1-predictions).T *(1-y) ))\n",
    "        #cost_history.append(cost) \n",
    "        w_history.append(w)     \n",
    "            \n",
    "    return w_history                  # return optimized parameters and cost history"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. 计算损失函数值\n",
    "\n",
    "对于一个$w$和一组样本$(X,y)$，可用下列函数计算损失函数值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_logistic(w,X,y,reg=0.):   \n",
    "    f = sigmoid(X @ w[1:]+w[0])\n",
    "    loss = -np.mean((np.log(f).T * y+np.log(1-f).T *(1-y) ))\n",
    "    loss += reg*( np.sum(np.square(w)))\n",
    "    return loss\n",
    "\n",
    "def loss_history_logistic(w_history,X,y,reg=0.):\n",
    "    #X = np.hstack((np.ones((X.shape[0], 1), dtype=X.dtype),X)) \n",
    "    loss_history = []\n",
    "    for w in w_history:\n",
    "        loss_history.append(loss_logistic(w,X,y,reg))\n",
    "    return loss_history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w: [11.3920102  -0.55377808 -0.83931251]\n",
      "[0.6577262444936194, 0.2267463703642394, 0.15646446608041154, 0.12698570286225017, 0.11034864425987873, 0.09949355960364478, 0.09177469381378581, 0.08596435646154407, 0.08141010065377202, 0.07773089221384288]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2553a44ffd0>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAed0lEQVR4nO3deXRc5Z3m8e+vdu2yNluWN9mYYLM4gNpgliSdhGCyOZkwibOSDgzDdNPbnJk0mT6TOXN6zunJ0JNO0qGbcICkJ5nEIcuAm0CchJCmExOwnMbGxhbYBiN5kyxb+1JV0jt/1JUoC9kq2yVf1a3nc45O3fvet6p+rzBPXb13KXPOISIihS/kdwEiIpIfCnQRkYBQoIuIBIQCXUQkIBToIiIBEfHrjevq6tyyZcv8ensRkYK0ffv24865+um2+Rboy5Yto7W11a+3FxEpSGZ28HTbNOUiIhIQCnQRkYBQoIuIBIQCXUQkIBToIiIBoUAXEQkIBbqISEAUXKC3He3nyz9r4/jAqN+liIjMKQUX6K909vO1X+7jxGDS71JEROaUggv01Ng4AAOjaZ8rERGZWwou0L+/rR2Ah379qs+ViIjMLQUX6Ml0Zg+9bzjlcyUiInNLwQV6yAyAcX0XqojIKQou0L08Z3zc3zpEROaawgt0Monu0B66iEi2ggv0FQ3lAJTHoz5XIiIytxRcoN96dRMAv9hzzOdKRETmloIL9HmlMb9LEBGZkwou0JfXZ6Zcbr+h2edKRETmloILdICashij6TG/yxARmVMKMtBLY2GGkgp0EZFsBRnoZbEIg7qXi4jIKQoy0KtKovTq0n8RkVMUZKDPK4tyclCBLiKSrTADvTTGiSHdD11EJFtOgW5m682szcz2mdk9p+nzDjN7wcx2m9k/57fMU80ri9EzlMTpBl0iIpMiM3UwszBwH3AT0AFsM7PNzrmXsvpUA38PrHfOvW5mDbNVMEBNaYzUmGNgNE1FQrcAEBGB3PbQ1wL7nHMHnHNJYBOwYUqfTwA/ds69DuCc68xvmaeaV5a5WlRfQyci8oZcAr0JaM9a7/Dasl0MzDOzX5nZdjP7zHQvZGZ3mlmrmbV2dXWdW8VAbXkm0LsV6CIik3IJdJumberkdQS4GngfcDPwX83s4jc9ybkHnHMtzrmW+vr6sy52Qq23h949oEAXEZkw4xw6mT3yxVnri4DD0/Q57pwbBAbN7BlgDfByXqqcorY8DkD3wOhsvLyISEHKZQ99G7DSzJrNLAZsBDZP6fMYcKOZRcysFLgG2JPfUt8wuYeuKRcRkUkz7qE759JmdjewBQgDDzvndpvZXd72+51ze8zsp8BOYBx40Dm3a7aKTkTDlMcjHNceuojIpFymXHDOPQE8MaXt/inr9wL35q+0M6stj2kOXUQkS0FeKQqZaZfuQe2hi4hMKNxAL49rD11EJEvhBnpZTAdFRUSyFG6gl8c4MZhkfFz3cxERgUIO9LI4Y+NO90UXEfEUbqBPXv6vA6MiIlDAgV7nXS16XAdGRUSAAg70iT10XVwkIpJRsIFe7+2hd/Yp0EVEoIADfV5pjGjY6OxXoIuIQAEHeihk1JfH6ewf8bsUEZE5oWADHaC+MkGX9tBFRIACD/T5FXGO9WkPXUQECjzQGyrjmkMXEfEUdqBXJOgZSjGaHvO7FBER3xV4oGdOXdQ8uohIgQf6/MoEAMd0LrqISGEHev3kHroOjIqIFHSgN1R6V4tqykVEpLADvbYsTjhkuvxfRIQCD/RwyKgrj+lcdBERCjzQIXPqoqZcREQCEOjzK3W1qIgIBCDQF1QlOKpAFxEp/EBfWF1Cz1CKwdG036WIiPgqp0A3s/Vm1mZm+8zsnmm2v8PMes3sBe/ni/kvdXoLq0oAONI7fKHeUkRkTorM1MHMwsB9wE1AB7DNzDY7516a0vVfnHPvn4Uaz2hhdSbQD/eMcFFDxYV+exGROSOXPfS1wD7n3AHnXBLYBGyY3bJyt7A6c/n/4R7toYtIccsl0JuA9qz1Dq9tqnVmtsPMnjSzS6d7ITO708xazay1q6vrHMp9s/mVCczgcK8OjIpIccsl0G2aNjdl/XfAUufcGuDvgEeneyHn3APOuRbnXEt9ff3ZVXoa0XCI+RUJ7aGLSNHLJdA7gMVZ64uAw9kdnHN9zrkBb/kJIGpmdXmrcgaN1QkdFBWRopdLoG8DVppZs5nFgI3A5uwOZrbAzMxbXuu9bne+iz2dhdUlHO7RlIuIFLcZz3JxzqXN7G5gCxAGHnbO7Tazu7zt9wO3Av/BzNLAMLDROTd1WmbWLKxK8IuXjuGcw/tcEREpOjMGOkxOozwxpe3+rOWvA1/Pb2m5W1hdwmh6nBODSWrL436VISLiq4K/UhROPRddRKRYBSPQvatFD+lMFxEpYoEI9MU1mUDvODnkcyUiIv4JRKBXlUSpSER4/YQCXUSKVyAC3cxYWlvKwW4FuogUr0AEOsCSmlLatYcuIkUsQIFeRvvJIcbGL9jp7yIic0qAAr2U1JjTtxeJSNEKTKAvrS0F4GD3oM+ViIj4IzCBvqQmE+iaRxeRYhWYQG+sShAJmc50EZGiFZhAj4RDNM0r0bnoIlK0AhPokJl2UaCLSLEKVKAvrS3l1eODXMA794qIzBmBCvQV9eX0j6Q5PpD0uxQRkQsuUIF+UUM5APs6B3yuRETkwgtUoK+ozwT6/i4FuogUn0AF+oLKBKWxsAJdRIpSoAI9FDKW15exv0tXi4pI8QlUoENm2mW/5tBFpAgFLtAvqi/nUM8wQ8m036WIiFxQgQv0Fd6ZLgc07SIiRSZ4ga4zXUSkSAUu0JfVlRIOGa8cU6CLSHHJKdDNbL2ZtZnZPjO75wz9fs/Mxszs1vyVeHbikTDNdWXsPdrnVwkiIr6YMdDNLAzcB9wCrAY+bmarT9PvS8CWfBd5tlY1VrLnSL/fZYiIXFC57KGvBfY55w4455LAJmDDNP3+GPgR0JnH+s7JqsYKDvUM0zuc8rsUEZELJpdAbwLas9Y7vLZJZtYEfBi4P3+lnbtVCyoBaDuqvXQRKR65BLpN0zb1/rRfAf7COTd2xhcyu9PMWs2staurK9caz9qqxkyg7zmieXQRKR6RHPp0AIuz1hcBh6f0aQE2mRlAHfBeM0s75x7N7uScewB4AKClpWXWblo+vzJOdWlUB0ZFpKjkEujbgJVm1gwcAjYCn8ju4Jxrnlg2s28Bj08N8wvJzFi1oJKXdGBURIrIjFMuzrk0cDeZs1f2AI8453ab2V1mdtdsF3iuVjVW0na0j7FxfXuRiBSHXPbQcc49ATwxpW3aA6DOuc+ef1nnb1VjBSOpcQ50DbByfoXf5YiIzLrAXSk64a2LqwHY0dHrcyUiIhdGYAN9eX055fEIOzt6/C5FROSCCGygh0PGZU2V7GhXoItIcQhsoAOsWVzNS0f6GE2f8fR4EZFACHSgv3VRNakxp/u6iEhRCHSgr5k4MKppFxEpAoEO9MaqBHXlcQW6iBSFQAe6mXHlkmq2v37S71JERGZdoAMd4JrmGg52D3Gsb8TvUkREZlXgA31tcw0Az796wudKRERmV+ADfXVjJWWxsAJdRAIv8IEeCYe4elmNAl1EAi/wgQ6ZefS2Y/30DCX9LkVEZNYURaBrHl1EikFRBPoVi6pIRENs3d/tdykiIrOmKAI9Hglz7fJannll9r7HVETEb0UR6AA3rqznQNcg7SeG/C5FRGRWFE2gv/3iOgDtpYtIYBVNoK+oL2dhVYJnXlagi0gwFU2gmxlvu7ierfu6SY2N+12OiEjeFU2gA7zt4nr6R9NsP6ibdYlI8BRVoN+4so5YOMTPdh/zuxQRkbwrqkCvSES5/qJatuw+inPO73JERPKqqAId4OZLF3CoZ5jdh/v8LkVEJK+KLtDfvXo+IYOf7T7qdykiInmVU6Cb2XozazOzfWZ2zzTbN5jZTjN7wcxazeyG/JeaH3XlcVqW1bBF8+giEjAzBrqZhYH7gFuA1cDHzWz1lG5PAWucc28FPgc8mO9C82n9pQtoO9bPK8f6/S5FRCRvctlDXwvsc84dcM4lgU3AhuwOzrkB98ZRxjJgTh9xfP+aRkIGj75wyO9SRETyJpdAbwLas9Y7vLZTmNmHzWwv8BMye+lzVkNFghtX1vPovx5mfHxOf/aIiOQsl0C3adrelILOuf/nnLsE+BDwV9O+kNmd3hx7a1eXv5fgf/jKJg71DLPtNd0jXUSCIZdA7wAWZ60vAg6frrNz7hlghZnVTbPtAedci3Oupb6+/qyLzaf3XDqf0lhY0y4iEhi5BPo2YKWZNZtZDNgIbM7uYGYXmZl5y1cBMWBOf5tEaSzC+ksX8PjOIwwl036XIyJy3mYMdOdcGrgb2ALsAR5xzu02s7vM7C6v20eAXWb2ApkzYj7mCuBSzI1rl9A/kuafdpz2Dw4RkYJhfuVuS0uLa21t9eW9JzjnuPkrzxCPhPmnP56zp86LiEwys+3OuZbpthXdlaLZzIxPXbuUFw/1sqO9x+9yRETOS1EHOmTOdimNhfnObw/6XYqIyHkp+kCvSET50JVNPLbjMF39o36XIyJyzoo+0AHuuKGZ1Ng439r6qt+liIicMwU6sLy+nPWXLuDbzx5kYFSnMIpIYVKge+56+wr6RtJ877nX/S5FROScKNA9axZXs255LQ/++gAjqTG/yxEROWsK9Cx/+u6VHOsb1RkvIlKQFOhZrl1ey40r67jv6X30j6T8LkdE5Kwo0Kf4zze/hZNDKR76tc54EZHCokCf4opF1dxy2QIe/JdX6ewf8bscEZGcKdCn8fn1l5BMj/OlJ9v8LkVEJGcK9Gk015Vx+43N/Oh3HWw/eNLvckREcqJAP427f/8iFlQm+G+bdzGmr6kTkQKgQD+NsniE//K+Vew61Me3tr7mdzkiIjNSoJ/BB65o5J2XNHDvlr28enzQ73JERM5IgX4GZsZf/5vLiYVDfP6HOxjX1IuIzGEK9BnMr0zwxQ9cyrbXTvLwb3RuuojMXQr0HHzkqiZuWj2fL/10r77ZSETmLAV6DsyMe2+9goaKBH/03d/RO6zbAojI3KNAz1F1aYy/+8SVHO0d4fM/3IFfX64tInI6CvSzcNWSedxzyyVs2X2Mr/ziFb/LERE5RcTvAgrN7Tc003a0n68+9QorGsr54JqFfpckIgJoD/2smRn/48OXsXZZDf/pBzv43eu6NYCIzA0K9HMQj4S5/9NX01iV4A++uY29R/v8LklEJLdAN7P1ZtZmZvvM7J5ptn/SzHZ6P1vNbE3+S51baspifOf2a0hEQ3z6oed5TVeSiojPZgx0MwsD9wG3AKuBj5vZ6indXgXe7py7Avgr4IF8FzoXLa4p5Tu3X0N6bJxPPvgc7SeG/C5JRIpYLnvoa4F9zrkDzrkksAnYkN3BObfVOTcxmfxbYFF+y5y7Vs6v4P987hr6R1J89BvPcqBrwO+SRKRI5RLoTUB71nqH13Y6twNPTrfBzO40s1Yza+3q6sq9yjnu8kVVbLpzHcn0OB/9xrPsOaI5dRG58HIJdJumbdqraszs98kE+l9Mt90594BzrsU511JfX597lQVg9cJKHrlrHdFwiI9941m27j/ud0kiUmRyCfQOYHHW+iLg8NROZnYF8CCwwTnXnZ/yCsuK+nIe+ffrmF+Z4DMPPc8j29pnfpKISJ7kEujbgJVm1mxmMWAjsDm7g5ktAX4MfNo593L+yywci2tK+dEfXse6FbV8/kc7+esn9ugbj0Tkgpgx0J1zaeBuYAuwB3jEObfbzO4ys7u8bl8EaoG/N7MXzKx11iouAJWJKA9/9vf45DVL+MYzB/jUg8/R2T/id1kiEnDm102mWlpaXGtr8HP/kdZ2vvjYLioSUb628UrWraj1uyQRKWBmtt051zLdNl0pOss+2rKYR//oeioSET754G+5d8teRtNjfpclIgGkQL8ALllQyea7b+AjVy3ivqf3s+Hrv2H34V6/yxKRgFGgXyDl8Qj3/ts1PHRbC92DSTZ8/Td8+ecvM5LS3rqI5IcC/QJ716r5/PzP38YH1izka0+9wnv+9hl+ufeY32WJSAAo0H1QXRrjbz/2Vr57xzXEIiE+961W7vjHVt3gS0TOiwLdR9ddVMcTf3IjX7jlErbuP867v/zPfPGxXXT1j/pdmogUIJ22OEd09o3w1adeYdO2duKREHfcuJw7bmymMhH1uzQRmUPOdNqiAn2OOdA1wP/+2cv85MUjVCQi3LZuGX9w/TJqy+N+lyYic4ACvQDtOtTLfU/v46e7jxKPhPj42iX8uxuXs7C6xO/SRMRHCvQCtq+zn3/41QEefeEQADdfOp/b1i1jbXMNZtPdCFNEgkyBHgAdJ4f49rMH2bStnd7hFJcsqOC265bxwTULKYtH/C5PRC4QBXqADCfH2LzjEN/aepA9R/ooi4V57+WN3Hr1Iu21ixQBBXoAOefYfvAkP2jt4PGdhxlMjrGkppSPXLWID125kKW1ZX6XKCKzQIEecEPJND/ddZQfbu9g6/7Md4tc1lTJey9v5H2XNyrcRQJEgV5EOk4O8eSLR3n8xSPsaO8BMuF+y2WN3LR6PisbyjUtI1LAFOhFqv3EEE/uOsJPXjw6Ge5N1SW885IG3rmqgXXLa0lEwz5XKSJnQ4EuHOkd5um9Xfxybye/2Xec4dQYiWiI61fUcf1FmZ+L52vvXWSuU6DLKUZSY/z2QDdP7+3kVy93cbB7CIC68hjXLq/luhV1XLeilqW1pQp4kTnmTIGuE5iLUCIa5h1vaeAdb2kAMvPuz+7vZuv+brbuP87jO48A0FiV4Kql82hZOo+rl85jVWMl0bDu5yYyV2kPXU7hnOPA8UG27jvO86+d5HcHT3KoZxiARDTEmkXVXO0F/BWLqqmv0D1mRC4kTbnIeTnSO8z2gyfZfjAT8LsP95Eez/y7WVCZ4LKmKi5vquLyRZVc1lRFQ0XC54pFgktTLnJeGqtKeP8VJbz/ioVA5mrVnR09vHiol12HennxUC9P7T3GxL7B/Mo4lzdVsaqxkovnV3DJggqW1ZVpukZklinQ5ayVxMJcs7yWa5bXTrYNjKZ56XDfKSH/dFsXY96efCwcYnl9GW9ZUDEZ8hfPr6CpuoRQSAdeRfJBgS55UR6PsLa5hrXNNZNtI6kxDnQN0nasj7ajA7Qd7aP1tZM89sLhyT4l0TDL6spYXl/G8snHcprry/TlHiJnSYEusyYRDbN6YSWrF1ae0t43kuKVY/3sPdrPga5BDnQNsPtQL0++eITxrEM6deXxyaBfWlvGkppSFteUsKSmlKqSqE6pFJkip0A3s/XAV4Ew8KBz7n9O2X4J8E3gKuAvnXN/k+9CJTgqE1GuXlrD1UtrTmlPpsd5/cRgJuSPZ4L+1eOD/PylY3QPJk/pWxGPsKimlCU1JSyeV8qS2lIWzytlcU0pi+aV6ApYKUozBrqZhYH7gJuADmCbmW12zr2U1e0E8CfAh2alSikKsUiIixoquKih4k3bBkbTtJ8Y4vUTQ7RP/JwcZn/XIL9q62I0PX5K/5qyGI1VCRqrSlhYnWBBVYKFVSU0ViVYWF3C/MoEsYgO0kqw5LKHvhbY55w7AGBmm4ANwGSgO+c6gU4ze9+sVClFrzweYVVjJasaK9+0bXzccXxgNBP2J4foODHM4d4RjvQO03FyiOdf7aZvJH3Kc8wyUzqZ0E+woDJBQ2WC+oo4DRVx7zFBbVlMB22lYOQS6E1Ae9Z6B3DNubyZmd0J3AmwZMmSc3kJkTcJhYwGL5BbltVM22dwNM2R3mEO94xwtHeEw73DHOnJPO7vGmTr/m76p4Q+QDhk1JXHJgO+ISvw6ysyHwC1ZTFqy2OUxyOa1xdf5RLo0/0LPaerkZxzDwAPQObConN5DZFzURaPnHY6Z8JIaozOvlG6Bkbo7Buls3+Uzv4Ruvozy0d7R9jZ0Uv34CjTXY8XC4eoKYtR4wV8bVmMmrI4teWxyfa68kxbTVmMyoQ+ACS/cgn0DmBx1voi4PBp+ooUrEQ0zJLazAHWM0mPjXNiMEln/yhd/aN0DyY5MZh57B5IcmIwSfdgkte6BzkxkGQwOTbt60TDxrzSGPNKY1SVRqkuiVJdGqWqJEp1acx7jFJdEstqj+ovATmtXAJ9G7DSzJqBQ8BG4BOzWpXIHBYJhyaneHIxkhrLhP5Aku7BUU4MvhH6JwaS9Awn6RlK8fqJIXZ2pOgZTjKSGj/t64VDRnVJNOtDIEZ1SZTKkigViYj3E6UyMWW9JEJlIko8EtIHQkDNGOjOubSZ3Q1sIXPa4sPOud1mdpe3/X4zWwC0ApXAuJn9GbDaOdc3i7WLFIRENExTdQlN1SU5P2ckNUbfcIqe4RQ9Qyl6hpL0DKfoHUpNfgBMrHf2j/DysX76hlMMjKZPOZd/OtGwUeGF/dTQz26rTEQpT0QojYUpj0coi0coj2fWy+IRfTDMQbo5l0iAOOcYTGY+DPpH0vSPZB77RlL0Za1Ptk/2y7T1jaQZGH3zweHpREJGWTxCmRfw2YE/8QGQvX2irTTubY9l+pbGwpTEwpREw0R0v58Z6eZcIkXCzCj3wvNcjY07BkYzAT84OsbAaJpB72dgNM1Q8tS2weTY5LbB0TRd/aMMJie2j5EcO/300VSxcGgy3LODvmQi+KNhSrwPgon2qX1LY5FT2ktjYRKxMKVF8IGhQBeRU4RDRlVJ5iBsPiTT417wp9/0ATGUHGM4NcZwcixr+c3t/SNpOvtGGUqlGU6OZ/qkxqY92+hMIiEjEQ2TiIaIR8LEoyESkcx6pt1b9rbFI1lt0TCJSObxjed5y9Hw5OvEs/olomHCF/A6BgW6iMyqWCRELBJjXlksr6/rnGM0Pc6wF/5DybGs5fS07SOpMUZS44ykM8ujqfHMYzrz2DeSymz3+o16287mr4ypomHzPiDCxCMh4tEQn1i7hDtuXJ7H30aGAl1ECpKZTe4Fz5vl9xobd4ymx7LC3gv89BsfEKOpU7dnPiTe+PAYSY2TTGeeU1c+O9/0pUAXEZlBOGSUxiKU5vePjLwL9hECEZEiokAXEQkIBbqISEAo0EVEAkKBLiISEAp0EZGAUKCLiASEAl1EJCB8u9uimXUBB8/x6XXA8TyWUwg05uKgMReH8xnzUudc/XQbfAv082Fmrae7fWRQaczFQWMuDrM1Zk25iIgEhAJdRCQgCjXQH/C7AB9ozMVBYy4OszLmgpxDFxGRNyvUPXQREZlCgS4iEhAFF+hmtt7M2sxsn5nd43c958rMFpvZ02a2x8x2m9mfeu01ZvZzM3vFe5yX9ZwveONuM7Obs9qvNrMXvW1fM7ML9yWG58DMwmb2r2b2uLce6DGbWbWZ/dDM9nr/vdcVwZj/3Pt3vcvMvmdmiaCN2cweNrNOM9uV1Za3MZpZ3My+77U/Z2bLZizKOVcwP0AY2A8sB2LADmC133Wd41gagau85QrgZWA18L+Ae7z2e4AvecurvfHGgWbv9xD2tj0PrAMMeBK4xe/xzTD2/wh8F3jcWw/0mIF/BO7wlmNAdZDHDDQBrwIl3vojwGeDNmbgbcBVwK6stryNEfhD4H5veSPw/Rlr8vuXcpa/wHXAlqz1LwBf8LuuPI3tMeAmoA1o9Noagbbpxgps8X4fjcDerPaPA9/wezxnGOci4CngnVmBHtgxA5VeuNmU9iCPuQloB2rIfM3l48B7gjhmYNmUQM/bGCf6eMsRMleW2pnqKbQpl4l/KBM6vLaC5v0pdSXwHDDfOXcEwHts8LqdbuxN3vLU9rnqK8DngeyvUQ/ymJcDXcA3vWmmB82sjACP2Tl3CPgb4HXgCNDrnPsZAR5zlnyOcfI5zrk00AvUnunNCy3Qp5s/K+jzLs2sHPgR8GfOub4zdZ2mzZ2hfc4xs/cDnc657bk+ZZq2ghozmT2rq4B/cM5dCQyS+VP8dAp+zN688QYyUwsLgTIz+9SZnjJNW0GNOQfnMsazHn+hBXoHsDhrfRFw2KdazpuZRcmE+f91zv3Yaz5mZo3e9kag02s/3dg7vOWp7XPR9cAHzew1YBPwTjP7DsEecwfQ4Zx7zlv/IZmAD/KY3w286pzrcs6lgB8D1xHsMU/I5xgnn2NmEaAKOHGmNy+0QN8GrDSzZjOLkTlQsNnnms6JdyT7IWCPc+7LWZs2A7d5y7eRmVufaN/oHfluBlYCz3t/1vWb2bXea34m6zlzinPuC865Rc65ZWT+2/3SOfcpgj3mo0C7mb3Fa3oX8BIBHjOZqZZrzazUq/VdwB6CPeYJ+Rxj9mvdSub/lzP/heL3QYVzOAjxXjJnhOwH/tLves5jHDeQ+fNpJ/CC9/NeMnNkTwGveI81Wc/5S2/cbWQd7QdagF3etq8zw4GTufADvIM3DooGeszAW4FW77/1o8C8Ihjzfwf2evV+m8zZHYEaM/A9MscIUmT2pm/P5xiBBPADYB+ZM2GWz1STLv0XEQmIQptyERGR01Cgi4gEhAJdRCQgFOgiIgGhQBcRCQgFuohIQCjQRUQC4v8DSvHbD1IgvF0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "reg = 0.0\n",
    "alpha=0.01\n",
    "iterations=10000\n",
    "w_history = gradient_descent_logistic_reg(X,y,reg,alpha,iterations)\n",
    "w = w_history[-1]\n",
    "print(\"w:\",w)\n",
    "\n",
    "loss_history = loss_history_logistic(w_history,X,y,reg)\n",
    "print(loss_history[:-1:len(loss_history)//10])\n",
    "plt.plot(loss_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. 决策曲线\n",
    "\n",
    "对一组{$x_1$}，根据$\\pmb w$算出其对应的{$x_2 = -w_0/w_2-w_1*x_1/w_2$}，然后在$(x_1,x_2)$坐标平面上绘制这些点对应的决策直线，可以看出学习到的模型可以很好地区分这2个类别的样本。绘制的代价曲线也反应了算法是逐渐收敛的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAEYCAYAAACgOtfQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeZxT5fX48c+ZjRkQEIYB2UFRkV0WxSJfFUSRqqjVWkXrVlHcaqtWq1aliuJWlopFBZUKSl1wQcWNVvHnDigO4IKowCgKgoDowGzn98dNIJPJnpvcTOa8X6/7yuTm5t6TgUlOnuc8zyOqijHGGGNMtsjxOgBjjDHGGDdZcmOMMcaYrGLJjTHGGGOyiiU3xhhjjMkqltwYY4wxJqtYcmOMMcaYrGLJjTHGGGOyiiU3xhhjjMkqltwYY4yJmYjcJiKXJ/C890WkZypiymQiskJEDvc6jobGkhtjjDEhiUhrEVER2ct3vwT4PXBfAqe7C/i7m/FFIyJfi8iR4e6n+noAqtpTVV9P1TVNaJbcmJBE5A4ReTrg/p0islBE8mN47jUi8mTQvikiMjXg/tUi8o2I/CQin4nI8FTGZIxJSF9go6p+57t/NvCiqpYncK7ngCNEpK1bwaWTiOR5HYOJnSU3Jpzbcd6I+onIhcBI4CRVrYzhuY8Bo0SkGYCI5AK/BR713d8fuAQYpKpNgaOBr1MckzEmfn2AjwPuHwO8EXiAiGwXkY6+ny/0tfS08d2/SkRmAKjqDmAJcFTwRaJ9IUrky1CIazwCdALm+2L+i4i0E5GnRGSjiHwlIpcFPedr37U/Bn4WkTxfrKt9sawUkRPDnT/gHEf6fj5ARF4XkS2+7qrjg651pYh8LCJbReQ/IlIY7+s0Pqpqm20hN+AmnDe2NUBH377mwPvAdqBXhOf+P+D3vp9HAKsDHusGbACOBPKTjcm3/xDgHZw33sfiPa9tttlWdwMeBv4RcH8jzpeSwGPWAT0AAUqBVUB33/0vgL4Bx04NPF/A/s7AL0Az3/1cYD0wGNjfd412vse6APvEGP/XwJGh7uN8uV8C3AAUAHsDXwJHBx3/EdARKPLtOwVo53v+qcDPQNtQ1wvcB+T7fh/X+q43DPgJ2D/guPd9524JfAJc6PX/gfq6WcuNieRDoDfwV1Vd59v3C/Br4Mmwz3I8Cpzm+/l0330AVPUL4HKcRGWDiMwVkXZJxAROsjNMVQ/DeYMaHeP5jDHhBbfc7InzgRxoC7AHTovMF8By33EjgW9UdVnAsT/5HqtFVdcAS4ETfLuGAb+o6rtANdAI6CEi+ar6taquTvaFAYOAElX9u6pWqOqXwAPA74KOm6qq69TXFaeqT6jqt6pao6r/wUnmDorheoNxfk8Tfdf7L/A8u98n/df6VlU3A/OBfsm9xIbLkhsTkoj0Bv4FzALO9e9X1UpV3RjDKZ4ADheRDsCJBCQ3vvM8qqqH4nxjU5wup4Ri8p3vW91dB1AF1MQQozEmDF938gFAYHLyI9A06NAfcT60LwemANuAFsBFOC01gZriJEOhhPxClOSXoUg6A+18XURbRGQLTqtKm6DjAr9EISK/F5GPAp7TC2gVw/XaAetUNfC9aQ3QPuD+dwE//4LzezUJsOTG1CEi7XG+NVyI8wbVO96hjL4E6HXgIeArVf0k4Pz7i8gwEWkE7ADKcb6dJR2TiHTFqQt4Pp54jTF17AfkASsD9n3s2x9oCzAQp2vmdZzkpi/Oh/4zQccGJ0uBwn4hSuTLUBga8PM6nPemPQO2pqo6KtxzRKQzTuvOJUCxqu6J01IlIc4f7Fugo4gEfu52Ar5J8LWYCCy5MbX4ioBfxOkXf05VfwHuBCYkcLpHcfqaHw3a3wiYCPyA802lNc43JkRkgYhcm0hMvuNmAWeqakUC8RpjdusDfK6qOwP2vQgcFnTcj8Cf2N1Ksw34I3Cfqu760uL7MjMAeDXUxcJ9IUrky1AE3+PU1oBT37LNVzBcJCK5ItJLRAZFeH4TnARmoy+2c3CSuFDnD/YeTn3OX0Qk3/fl7DhgboKvxURgyY2pRVW3qWpfVZ0asO8uVR2SwLkeUVVR1TuD9n+sqgf5viW1VNVjVfVb32PHqOqt8cbkG6b5GHCTqn4Wb6zGmDp6U7eV5d84IyGLAvb9iNPC4/8Ssw2nruaBoOceD7zu/1sPI9QXori+DEVxG3C9rzvpTzjJRT/gK9/5Z+AMmghJVVcCd+MMXvge53f0Vqjzi8iVQc+twPkdHOO71r04gy4+jSN+EyNRjdSKZkxdIvIizhvCGpxvZw97GxGIyJnAJJwmYoB/+Yr9jDEJEJH/AY+q6gNB+28FNqjq5DjP9x5wnqouj3qwMUmy5MYYY0wtIjICpyW0u6r+4HU8xsTLZlw0xhizi4iU4tS0nGyJjamvrOXGGGOMMVnFCoqNMcYYk1XqVbdUq1attEuXLl6HYYwJsGTJkh9UtcTrONxi7zPGZJ5432fqVXLTpUsXFi9e7HUYxpgAIrLG6xjcZO8zxmSeeN9nrFvKGGOMMVnFkhtjjDHGZBVLbowxxhiTVSy5McYYY0xWseTGGGOMMVmlXo2WMiaaitJSdixciG7dijRvTuHw4RT07u11WMYYY9LIWm5M1qgoLaV8/nx061YAdOtWyufPp6K01OPITNZZuxY++sjrKIwxYVhyY7LGjoULobKy9s7KSme/MW66/XYYMcLrKIwxYVhyY7KGv8Um1v3GJCwvD6qqvI7CGBOGJTcma0jz5nHtNyZhltwYk9EsuTFZo3D4cMjPr70zP9/Zb4yb8vMtuTEmg9loKZM1/KOikhktZaOtTEys5caYjGbJjckqBb17J5yM+Edb+YuS/aOt/Oc1Zhd/cqMKIl5HY4wJYt1SxvjYaCsTszzf98Lqam/jMMaElPLkRkQeFJENIrI8aP+lIvKZiKwQkTtSHYcx0dhoKxOzdeuc2507vY3DGBNSOlpuHgZGBu4QkSOA0UAfVe0J3JWGOIyJyEZbmZh9/LFzu3att3EYY0JKec2Nqi4SkS5Bu8cBE1V1p++YDamOw9RfyRT5hntuqP2Fw4fXqrkBbLSVCe2yy+Ddd63expgM5VVB8X7AUBGZAOwArlTVDzyKxWSwZIp8wz23au1aKpctq7O/6LjjyO/bl8olS3YViub37ZtRxcQ2mitDFBU5t+Xl3sZhjAnJq4LiPKAFMBi4CnhcJPRXIBEZKyKLRWTxxo0b0xmjyQDJFPmGe27l4sUh95cvWEDlhx86iQ2AKpUffpgxa1PZ2lkZxJ/c7NjhbRzGmJC8Sm7KgHnqeB+oAVqFOlBV71fVgao6sKSkJK1BGu8lU+QbdyFweXnd0S/V1exYsCC+86SIjebKIIWFzq213BiTkbxKbp4BhgGIyH5AAfCDR7GYDJZMka9bhcCaIR9gNporg+yxh3O7fbu3cRhjQkrHUPDHgHeA/UWkTETOAx4E9vYND58LnKXq7wswZrdkllQI+dxwYj3OQzaaK4P4f+eWWBqTkdIxWuq0MA+dkeprm/ovmSUVgp+LyO56mkAiFB13HOULFoTuZigqyohCXhvNVZuIjASmALnADFWdGOKYw4HJQD7wg6oe5srFmzVzbrdtc+V0xhh32fILJuMls6RC4HODR08BkJ9P0XHH7Tqm/JlnoKZm9+M5OeT37JkRyzK4sXZWthCRXGAaMAKnhu8DEXlOVVcGHLMncC8wUlXXikhr1wKwlhtjMpolN6bBiJYchHs8UiFvuhOLZBK9LHMQ8IWqfgkgInNxJgZdGXDM6TgDF9aCy/NpNWrkbJbcGJORLLkxDUq05CDU4+Xz5oU81gp5PdUeWBdwvww4OOiY/YB8EXkdaApMUdV/uxZB8+aW3BiToSy5MSYKad48ZCITTyFvJtTsZJlQ82IFF1TlAQOA4UAR8I6IvKuqn9c5mchYYCxAp06dYovAkhtjMpatCm5MFOFGXeXtu29Mz7fJ91KiDOgYcL8D8G2IY15S1Z9V9QdgEdA31MkSmk/LkhtjMpYlN8ZEUdC7N/l9634mVi5bFlOCYpPvpcQHwL4i0lVECoDfAc8FHfMszjIveSLSGKfb6hPXIthzT9iyxbXTGWPcY8mNMTGoWrWq7s4YExSbfM99qloFXAK8jJOwPK6qK0TkQhG50HfMJ8BLwMfA+zjDxZe7FkSLFrB5s2unM8a4x2pujIlBMgmKGzU7pi5VfRF4MWjf9KD7dwJ3piSAFi3gxx9TcmpjTHIsuTH1UtoLdIuKQk7wF0uCYpPvZSl/cuNbQd4YkzksuTH1TvBkfKmeVK+itBQqKuo+kJMTU4Jik+9lqZYtnf+Dv/wCTZp4HY0xJkBWJjc7d+6kUaNGXodhUiTdk+rtWLiw7mrhgDRqFPP1bPK9LNSihXP744+W3BiTYbIyuTnjjDMQEe655x5at3ZvxnXjncBuqHBSVaAbtt4mQ1YLNx7xJzebN0OHDt7GYoypJetGS3399dcsWLCAJ554gh49evDoo49iC47Xb8HzxISTqgJdW43bhBTYcmOMyShZ13LTpUsXli9fzvnnn89rr73GmDFjmDt3LtOnT6ddu3Zeh2cSELIbKliSBbqRCpTjKQi2mYgbEEtujMlYWddyA06C88orrzBjxgyaNWvG/Pnz6dGjBw8//LDXoZkExNJiE7iyd7yizSBc0Ls3Rccdt6ulJtz1bCbiBqZlS+fWkhtjMk7Wtdz4iQjnnXceI0eO5IILLuCFF15g2bJlXodlEhBpnphml1+e9PljKVCOpSA4k1YPN2lgLTfGZKyUt9yIyIMiskFE6swMKiJXioiKSKtUXb99+/bMnz+fxx9/nFtuuWXX/rVr11JTU5OqyxoXhVzbycV5YtyaQdhmIm5gmjaFnBybpdiYDJSOlpuHgXuAfwfuFJGOwAhgbaoDEBFOOeWUXfd/+uknhg4dSteuXZkxYwbdunVLdQgmCXXmiSkqQoHyefPYsXBhyLqWeGpf3JpB2K3zuFW3Y/U/KZaT46wvZS03xmSclLfcqOoiINRXm0nAX4C0D2X6/PPP2bFjB2+88QZ9+vRh0qRJVIeYx8RkjoLevWl2+eXkDxzoDMH2DcMOVdcSb+2LGy1DYSf6i/M8v7zwAuXz5iVdt2P1P2liSzAYk5E8KSgWkeOBb1Q1ahGMiIwVkcUisnjjxo2uXH/AgAGsXLmSM844g/Lycv785z9z6KGH8skn7i0YbNxXUVpK5eLFdR8IWsAy3lW4Yy0YjhRX+fz5dee9KSoKW3i8bfJkto4fz7bJk3clHLG+vljYSuRpYsmNMRkp7QXFItIYuA44KpbjVfV+4H6AgQMHutbKU1xczCOPPMKpp57KBRdcwLvvvku/fv2YPHky48aNc+syxkWRPpgDu4MSqX1JZgbhcEPVpaAg7IiqUEtHxPL6Yu1qsvqfNGnZ0pIbYzKQF6Ol9gG6AsvEWWyuA7BURA5S1e/SHcyxxx7LihUruOqqq5gxYwbt27dPdwgmRpE+mAPrWsLVvoCTHMRbnxPt8XgSiUgtKtFeXzxratlK5GnSogV89ZXXURhjgqS9W0pVS1W1tap2UdUuQBnQ34vExm/PPffkgQceYNmyZRx//PG79r/22mtUhKqjMJ6I9MEcWNcSqcYluHUkWm1KLLUr8cxgHCkRivb64ulqSvUIM+Nj3VLGZKR0DAV/DHgH2F9EykTkvFRfM1F9+vTZ9fPixYsZOXIkAwYMYHGoOgiTdiE/sIH8gQNrtVxE6l4KTi6iJQyxJBTxJBKREqFory+eFqJk64hMjPzdUjathDEZJeXdUqp6WpTHu6Q6hkRUV1fTtWtXli9fzsEHH8xVV13FjTfeSFFRkdehNVh1hoQ3b07evvtStWoVW8ePr9VlFGu3TKSEYdvkyTElFKHiClcLE2kph+DzIAKqVK1aRUVpadxdTbYSeRqUlDgrxm/ZsnvGYmOM57J2huJkHXzwwSxbtowbbriBSZMmcfvtt/PMM88wc+ZMhgwZ4nV4DVbgB3akGpRY14OKVJ8Ta41PcFzR4ofwiVBB795UrV3rjJryLfjqf135fftSuWxZ7deUm4tWVNRJ7kyatPLNP/rDD5bcGJNBsnJtKbc0btyYu+66i7feeosDDjiAzz77jKFDh/Lggw96HZoh+nIHsXTLhOsKiignJ6naFf+cPUUnnQQ4kxH6h4RHGg5etWpV7ddUVOQkQBHm/DEpVlLi3Lo0TYUxxh3WchODwYMH8+GHH3LzzTdz3333cfTRR3sdkiH6KKVYWlPqdAXFdOHkZyQI1+okeeH/JHXr1lqvadvkybsSm11sLav0suTGmIxkLTcxatSoEbfccgurV6/eNVy8urqaiRMnsm3bNo+ja5jiGaUUib8lJebnqSbdQhKu1anORIAB4qkXMmliyY0xGcmSmzg1a9Zs189Tp07lr3/9Kz179mTBggUeRtUwuT3cOa4uqiRn+00kAQlVLxSKzWWTRv6aG0tujMkoltwkYcSIEQwaNIiysjJGjRrF2WefzY8250VMwi1BEA+3hzvXOV+UkXHJtJCETUCKiiA3t87u4OHuYHPZZISiImjSxCkoNsZkDKu5SUKvXr14++23mTRpEn/729+YNWsWL7/8MtOnT2f06NFehxeXdK4gHc9Mu9FEqqtJ5DUFn2/r+PFhj02mhSTcaK78nj2pXLq09sE5OeR16hQyVohtCHo2EpGRwBQgF5ihqhODHj8ceBbwTyE8T1X/7nogJSXWcmNMhrHkJkl5eXlcddVVHH/88Zx33nm89dZbnHDCCcyfP59jjz3W6/Bi4mayEYtoo5zc4NZrijRUPNkRU1A3MdmxcGHdCeFqasL+bhrqXDYikgtMA0bgzHL+gYg8p6orgw59U1VT+4doyY0xGceSG5fsv//+LFq0iGnTpvHcc89xzDHHeB1SzNKRbARKRyFsvK8psJXHP3mef5LAOnPLELqbKNz5wrWohEpMyufNC3k+KxKu4yDgC1X9EkBE5gKjgeDkJvVKSuA7z1aPMcaEYDU3LsrJyeHSSy/llVdeIddXN1FWVsYZZ5zB+vXrPY4uvHSPuklHIWw8ryl4/ajAyfMqly0jv2/f2nU9J51E41//Ouy1Y1mPyn9ccN2RFQnHrD2wLuB+mW9fsENEZJmILBCRnuFOJiJjRWSxiCzeGG8rjLXcGJNxLLlJAd9q5wBceeWVzJkzhx49ejBr1izUhTlS3JbuD9R0FMLG85pCtvL4+SbPa3b55TS/8UaaXX551NasWNajCpcA5e27rxUJx0ZC7Av+41oKdFbVvsA/gWfCnUxV71fVgao6sMQ/vDtWJSVWUGxMhrHkJsXuvPNORo0axZYtWzj77LMZNWoUa9eu9TqsWlKVbIQbEZWORR3jeU3RWqjibcGKpdUoXAJUZxZiW/AynDKgY8D9DsC3gQeo6jZV3e77+UUgX0RauR5Jq1bOZIo//+z6qY0xibGamxTr2LEjzz//PLNnz+aPf/wjL730Er169eLOO+/k/PPPJyfH+/wyFaNuohX0proQNp7XFKlo2P94PGJZ4DJSAtRQi4Tj9AGwr4h0Bb4BfgecHniAiOwFfK+qKiIH4XyZ2+R6JIET+TVp4vrpjTHxs+QmDUSEM888kxEjRnDRRRfx9NNPc/HFFzN06FB69OjhdXiA+6NuyhcsSGuRciixvqaQw7L9EmjBimXRznhX+Da1qWqViFwCvIwzFPxBVV0hIhf6Hp8OnAyME5EqoBz4naaiXzgwuenSxfXTG2PiZ8lNGu2111489dRTPPnkk3z55Ze1EpuampqMaMVxQ0Vpad01j3y8HvUTaRRTqNFSibRgxdJqFOuq5SY8X1fTi0H7pgf8fA9wT8oD2Wsv59ZGTBmTMVKe3IjIg8CxwAZV7eXbdydwHFABrAbOUdUtqY4lE4gIp5xySq19zz77LHfccQczZ86ke/fuHkWWmFDJQqRlCbxsmYjUVRZImjVLulsuWqtRQ5+AL6u0bevcZvCISGMamnS03DyM8+3p3wH7XgX+6mtavh34K3B1GmLJOKrKzTffzJIlS+jXrx/jx4/niiuuIC/C6tCZImyyEG7kEclNfJessKOYFixAq6pqv45nnnH2l5enLPGw2pos0aaN09pnyY0xGSPl/SCqugjYHLTvFVWt8t19F2ekQ4MkIrz22muce+657Ny5k2uuuYbBgwdTmsSK0+kSLllAQo3SddZqSrZIOZn1qMIW8ZaX130dNTW7VugON0+NMYAzKq9VK0tujMkgmVDkcS4QdkntpCbXqif23HNPZs6cycsvv0ynTp1YsmQJAwYMYPz48VRUVHgdXlhh62dUQw/DTmLW5lgnxoskqS6xJFcBN1mubVv49tvoxxlj0sLT5EZErgOqgDnhjklqcq165qijjmL58uVcdNFFVFZW8tBDD7Fz506vwwor0kR5keZqSaQFJpaJ8aIpHD485IrbsYpUDO3GKuemHmvb1lpujMkgnhV2iMhZOIXGw1MyPLOeatq0KdOmTeO3v/3trvsAv/zyCzk5ORQWFnoZXi2RRvyEqydJdEFL15aISOK/WrhkLpHXlM5V2E0atG0Ly5d7HYUxxseTlhsRGYlTQHy8qv7iRQyZ7rDDDuOwww7bdf/aa6/lwAMP5J133vEwqtoSmWk40RYYN5aICLnidjjBLTwRhmnH+5rc6GIzGaZtW/j++9j/fxljUiodQ8EfAw4HWolIGXAjzuioRsCrvnWY3lXVC1MdS31VXl7Oq6++yqeffsqQIUO4/PLLueWWW2jcuLHXocU94ifRFhg35oWJtZUncEh7YMsKwLbJk+u0tsT7mtK9CrtJg7ZtoarKWWOqdWuvozGmwUt5cqOqp4XYPTPV180mRUVFLFmyhPHjx3PnnXcyadIknnvuOWbMmMHhhx/udXhxSXRmXjfmhYm2zAIQtlstUtdTvK8p3auwmzQInOvGkhtjPJcJo6VMDAoLC7ntttt477336NOnD6tXr+aII47goosuoqqqKvoJMkS0BS0jFeYW9O4d1+rcMV07JwcpKgIid6tFam2Jd+HRdK/CbtLAJvIzJqNk/kxxppYBAwbwwQcfMHHiRG655RZ++OGHejHhn1+kFphEi43duHY00Ra6jPSagvfb0gtZqF0759aSG2MyQv35VDS7FBQUcMMNN3DiiSfSpk2bXftXr15Ny5YtadGihYfRRReuTicVtSgVpaXOIp6+CfmkqIjCY46pc75oo5eidT2Fek3hkrWi446j6LjjbLRUNvG33NhcN8ZkBEtu6rHeAR+GlZWVnHLKKXz33XdMnz6d448/3sPIEpNILUqkpKSitJTyZ56pNYJFy8spnzeP8gULKPIlObG0GCXS2hIpWUukW81ksMJCKC6GsjKvIzHGYDU3WWPz5s0UFRWxfv16Ro8ezemnn84PP/zgdVhxibcWJdqQ6ohDv8vLdx0by1DuRIa9W+FwA9OpE6xd63UUxhgsuckabdq0YdGiRUyZMoXGjRvz2GOP0aNHDx5//HHqyxyJ8RbmRktKoiYRvmNjTULiLWi2wuEGpnNnS26MyRCW3GSR3NxcLrvsMkpLSzniiCPYuHEjp556Kuecc47XocUk3taRaElJLEmEvzsrlGSTkHiTNVPPdeoEa9YkNQu2McYdVnOThfbee29ee+01HnjgAa666ioOOeQQr0OKKrh2puikk2JqGYlU5Fs4fDjlTz8d8cPGX6cTSz1NrMXJfm7MzWPqkU6d4KefYOtW2HNPr6MxpkGz5CZL5eTkcMEFF3D88cfXGlH1/PPP069fPzp06OBhdLXFOgQ8OAHK23dfKpctq9s1VVGxe34ckfDJTcCEfRA5CQlbnPzss3XiDBTvDM5uszWs0qhTJ+d27VpLbozxmCU3Wa6tf4gq8NVXX3HqqaeSl5fHXXfdxR/+8Ad8y19ENGcOXHed857dqRNMmABjxrgXY7jamfKnn6Z83ryQiYxu3UrlsmXk9+1L5YoVu1pTYPeIqIhEatXnREtCwhYnV1dn7LIJqZ43yAQJTG769PE2FmMaOKu5aUCKiooYMWIE27ZtY+zYsYwYMYKvvvoq4nPmzIGxY3eXEqxZ49yfM8e9uMIW/vpaXHTrVioXLw6ZAFWtWoUUFCRw0d3nDhxhFW6G5EjFyZk6+inRRUpNggKTG2OMpyy5aUD22msvnn76aebOnUurVq1YuHAhvXr14p///Cc1YYZMX3cd/BK0bvsvvzj73ZJM4a5u3Zp8cuH7wI80tDxSjJk6+smGoqdZmzZOAbklN8Z4zpKbBkZEOPXUU1m5ciWnnnoqv/zyC5dddhmXXnppyOPDvU+7+f4dclRRrIqKXEkudOvW6OtH5YT4c8nNjWldLC/YUPQ0y8mBjh0tuTEmA1hy00CVlJQwd+5c5s2bR6dOnbjwwgtDHudvaY91fyL8Q8CJof4nmBBDcuQ7rzRvDr5FMuscEmHFcP/6UUUnnFDr+VJURNHo0bVmOQ43oaAXsn0ouoiMFJHPROQLEbkmwnGDRKRaRE5OeVD+4eDGGE9ZQXEDd+KJJ3LssceS7/sQVFWuv/56TjvtNHr16sWECU6NTWDXVOPGTlGx6xKYH0TLy3ePdlqwAA0oLAYgP7/WXDnBRbb+YwqHDw87oV+k9aP8UrEuVrKyeSi6iOQC04ARQBnwgYg8p6orQxx3O/ByWgLr2hVeeiktlzLGhJfy5EZEHgSOBTaoai/fvpbAf4AuwNfAb1X1x1THYkLLD/h2P2/ePG699VbuvPNO/va3v3HNNdcA+SkdLbUr4QhHxFm7JzhxoW7iEW3oc7QP/ERX687U+havh6Kn0EHAF6r6JYCIzAVGAyuDjrsUeAoYlJaounVzVgb/+Wdo0iQtlzTG1JWOlpuHgXuAfwfsuwZYqKoTfc3J1wBXpyEWE8WIESO44IILuO+++7jhhht46qmneOihh/j66wMTPme0hCNkq4efr+UFYks8YvkwD3dMpMQn2VXDjevaA+sC7pcBBwceICLtgROBYURJbkRkLDAWoFMyfa7dujm3q1fbcHBjPJTymhtVXQRsDto9Gpjl+3kWcEKq4zCxaY+m3H8AACAASURBVNasGdOnT2fhwoV07dqVZcuWMWjQIK6//np27twZ9/liqUWJ1Lrh71JKZOHKRIRaPyqW15Dt9S0ZKFSBVnC/5mTgalWtjnYyVb1fVQeq6sCSkpLEo/InN198kfg5jDFJ86rmpo2qrgdQ1fUi0tqjOEwYw4YNo7S0lOuuu46pU6cyYcIEmjZtytVXx9fAFkstSqRWj+AupUSSmWRm6a0oLQ29hEPAa6i1srhvRuRI17FZg11RBnQMuN8B+DbomIHAXN9Ela2AUSJSparPpCyqffZxbi25McZTGV9Q7FpzsYlbkyZNmDx5Mqeccgq33XYbl112WdzniKUWJda1ncKJlCwkM0vvrueGKXTWrVvrFiir1lnWIeQ5bdbgZH0A7CsiXYFvgN8BpwceoKpd/T+LyMPA8ylNbACaN4dWrSy5McZjXiU334tIW1+rTVtgQ7gDVfV+4H6AgQMH2nK7HhgyZAjPP//8rvtbt27lhBNO4KabbuKwww6L+NxYalGSGdUTLVlIZhRTxFog32uI5fyByVfIta4SGFXV0Ft/VLVKRC7BGQWVCzyoqitE5ELf49M9C65bN0tujPGYV8nNc8BZwETf7bMexWESMGnSJF5//XUOP/xwLr74Ym677TaaNm0a8thYW2US7XKKllwkMoqpVjISju81hFvDyv/ckC07ccYTKj5r/QFVfRF4MWhfyKRGVc9OR0yAk9y88UbaLmeMqSvlBcUi8hjwDrC/iJSJyHk4Sc0IEVmFM0/FxFTHYdxz7bXXcuONN5KXl8e0adPo3bs3r776ashjU10IHC15iXeW3uDi4dBPll2vIdr5o7X+RIsnFFszKsN16wZlZbBjh9eRGNNgpWO01Gmq2lZV81W1g6rOVNVNqjpcVff13QaPpjIZrKCggJtuuonFixfTv39/1qxZw1FHHcUf/vAHtmzZktZYIiUXFaWlUFFR98EI9TxRk5H8fIpOPHFXchZtlFRMLTJxjqrK1Dl1jE+3bk4L3erVXkdiTINlyy80UHPmQJcuznI4Xboktsp33759ee+997jtttsoKChg5syZnHHGu+TlOaUleXlwwak/pHRZgnDJRd6++zrXDZ74r6goYstRtBabwEU2IYaWqShLSiTSkmVrRmW4Hj2c25XB8wkaY9Il40dLGffNmVN7SYU1a5z7EP/Mw3l5eVxzzTWMHj2ac855lhdeGLnrserqndz/eDHVXw7n7mMDpqR3cVmCcMXI4VpgpKAg4nUjrTHlr5cJrnEJVy9UUVoacUmJ5jfeGPaxSJIdXWZSbP/9naR2xQo45RSvozGmQbKWmwbouutqrxUFzv3rrkv8nAcccACLFweuXfge0BWYx8NLBtY53s0ulFAT7yXadRPzCuVRalyiLSmRTCtLuiY0TBcRaeJbAyo7NG4Me+9tLTfGeMhabhqgtWvj2x+r6lrzwM4E1gMnU62/YcP2rrTeY49dj6a6CyXR5RBCtQQlkihFW1Ii2VaW+rxmlIjk4MxLMwZnWYSdQCMR2Ygz+ul+VV3lYYjJ69nTabkxxnjCkpsGqFMnpysq1P5k5OYGJjjTgQOBvwBPcfC0IiaOHMlv+/RBCgpi+nBPZi6XZLpughOHbZMnx50oxbKkRAP2P+A14K/AclWtgV0L6h4BTBSRp1V1tocxJqdnT3jxRaegvaDA62iMaXCsW6qBCCwg3r69bs9L48bOat/J8NftOHKAccByOrQ5nB/Ly7ng6af53RNPsGXw4JhnB060ENnNrpuwXVUVFWHjiVT028ATG4AjVfVm4HN/YgOgqptV9SlV/Q3wH+/Cc0GPHlBVBavqdwOUMfWVtdw0AMEFxJs2OV8mi4th82anxWbChPiLiYPde69ze//9TgtObi6MHduZadP+y8MPP8yf/vQn3l+/noIDDoh6rmRmFvZzq+vGf47yBQsgYPSVlpeHnTwvHUW/9XWWYlX1/1I+EJFXgH+p6hdhjqmfevZ0bleu3P2zMSZtYm65EZERIvKAiPTz3R8b7TkmM4QqIK6ogD32gJoa+Prr0IlNIsPF773X+cKq6tzeey+ICOeccw4rV67kySefpG3btgBUVVWxbt26kOfJtLlcCnr3RkJ1L4QpLE510W+yLVsZoi/wOjBJRF4QkWNFooydry+6d3f+cJYv9zoSYxqkeFpuLgLOAa739Y33S01Ixm2JFBC7OVzcr127drRr127X/X/84x/8/e9/5/bbb2fcuHHk5OzOtRMtCE6leBOuVBb9utGylQGaAyuA8UAf4A7gHqCLhzG5o6gI9tsPPvrI60iMaZDiqbnZqKpbVPVK4CicUQ6mHghXKBypgDgVw8UDqSorVqzg559/5pJLLuHwww9nVUB9QrSZf72QSZPnZVrLVoI2AY8AvwXa4yyQ+3dPI3JT//6wdKnXURjTIMWT3Lzg/0FVrwH+7X44JhUmTHAKhgNFKyBO1XBxPxFh1qxZPPnkk7Rp04Y333yTPn36cPfdd1NdXZ2Rc7lkUsKVSYlWEgYCnwO9gZXAVFV90NuQXDRggLPG1IYNXkdiTIMTNbkRkckiIqpaa+VuVf1n6sIybhozxiny7dzZmTi1c2fnfqTupURaexLxm9/8hpUrV/L73/+eHTt2cOWVV/KrX/2KjRs3hpycz0uZlHBlUqKVKFVdqqrn4Mx30w1YJCLXehyWe/r3d26t9caYtIul5mY78JyInKqqv4jIUcCNqjokxbEZF40ZE1+tzIQJtWtuwJ3h4qG0bNmSWbNmceqpp3LBBRdQUFBAcXGx+xdyQaZMnhdu2YlMiC1WIvI6sAfQGBCgBjgZuNXDsNxz4IHO7dKlMHJk5GONMa6Kmtyo6vUicjrwhojsBH4GronyNFPP+ROh665zuqKiDRefMyf2Y8MZNWoUy5cvZ9u2bbuKi9etW8emTZvo18/q14NlSqKVhLOBLcBW1QiLcNVXzZs7K4Rby40xaRdLt9Rw4HycpKYEuExV30x1YCYxbqz27TdmjDNM3D9cHEKf2z+yas0aZwi4f2RVItdu3rw5HTt2BJyi4/PPP59BgwZxww03sHPnzsRfjMkY/uHeqvq1b5BCncQma4aE9+8PS5Z4HYUxDU4sBcXXAX9T1cNxmoz/IyLDUhqVSYibSUY8507VyKrKykr2228/qqqquPnmmxkwYADvv/9+cic1meB/InKpiNSq4BKRAhEZJiKzgLM8is1dgwY53wy+/97rSIxpUKImN6o6TFX/n+/nUuAY4BY3Li4ifxKRFSKyXEQeE5FCN87bUKVy+Hakc6dqZFVBQQFTp05l0aJF7LvvvqxYsYJDDjmEv/zlL5QHzBRs6p2RQDXwmIh8KyIrReRLYBVwGjBJVR/2MkDX/OpXzu3bb3sbhzENTNxrS6nqeiDpIRki0h64DBioqr2AXJyVgk2C4kky4u2+inTuVI+sGjp0KMuWLeOqq64C4M4772TQoEFUVFS4cwGTVqq6Q1Xv9Q1K6IzzftJfVTur6vmqmj0z3w0YAI0awVtveR2JMQ1KQgtnqqpbX5vzgCIRycMZMfGtS+dtkMIlEy1b1r6fSPdVpAQmkXl04lVUVMQdd9zBO++8Q48ePTjppJMosNWW6yUR+YeInC0i/YEcVV2vqlu8jislGjWCgQMtuTEmzTxbFVxVvwHuAtYC63FGTLwSfJyIjBWRxSKyeOPGjekOs16ZMCH04tU//VQ7cYml+yq4ZWfUqPAJTCLz6CTqoIMOYunSpVx//fW79i1YsIDXXnut1nFuFlYb130BDAb+Caz3dUvNFZFrfWvYNfI4PncNGeIUFVtXqjHpo6qebEAL4L84I7DygWeAMyI9Z8CAAWoiKy5Wddpjam+dO+8+RiT0MSLO47NnqzZuXPuxxo1Vx42rff7iYudYL23atEnbtGmjgB5xxPnaseOWXa8lOH6vY81WwGJN7r2gK3A8cD0wD/gSODqZcyazuf4+8+yzzn/CN95w97zGNCDxvs941nIDHAl8paobVbXS96b2Kw/jyQqbN4feH1gzE61GJlzLzuOP1/7yuWlTfKOxKkpL2TZ5MlvHj2fb5MmurGDdtGlTLrvsMvLyCvjf/x5g3bqewAsEDy52c10s4y5V/UpVn1PVW1T1JGAIMUzkJyIjReQzEflCROrMvSUio0XkYxH5yNf6e2gq4o/KX1T8ps2gYUy6eJncrAUGi0hj35wWw4FPPIzHE253n8RSdxOtRiZc8fCmTYmPxqooLaV8/vxdCzvq1q2Uz5+fdIKTn5/PtddeS0nJh8DBwDfAscDvgdqZnlvrYpnUUmfQwqORjhGRXGAazujNHsBpItIj6LCFQF9V7QecC8xIQbjRtWoFfftCUNepMSZ1vKy5eQ94ElgKlPpiud+reLwQqbA30aRnwgQIVWe7bdvuc0SrkYl3lFMsScOOhQuhsrL2zspKZ78LvvuuB/AWcDdQiLPY9LHA7iYct9fFMqmjqndHOeQg4AtV/VJVK4C5wOigc2z3NWcDNCHwP0O6jRjhFBX//LNnIRjTkHjZcoOq3qiq3VW1l6qeqaoNagracN0/f/xj4pPxjRkDTZvW3V9ZWbuFJXj24cDi31AtO5H4k4ZICZm/xSZYuP3xcmLIBf4MfAwchjMdkzPRbarWxTKeaQ+sC7hf5ttXi4icKCKfAi/gtN6ElPKBC0cd5fwRLlrk/rmNMXV4mtw0dKno/oHY6m4iCW7Zyc0Nf6yIk3w1bQpnnBE+IfOvpF3n+WH2x6t2QrYv8D/8E2l37gwjRlyPyKNocDGOqa9CLc9Q5x9XVZ9W1e7ACcDN4U6mqver6kBVHVhSUuJimD6HHuoMC3/1VffPbYypw5IbDyXb/ROupcSNSfUCW3ZqasIf588Vtm+v+9gvv8BZZznx9b7jIp5Y0bf2Afn5FA5Pej7IXfHW7moTHnnEiW/evKU8++wExowZw+jRo/nmm29cuabxVBnQMeB+ByLMk6Wqi4B9RKRVqgMLqagIhg615MaYNLHkxkPhCnuLi0MfH5ichKvXueii0IlGMt0yydSqVFc78a1dX8Af5x/PE6sOApwWm6LjjnN1VetwXW0HHnggM2bMoFmzZsyfP5+ePXvy4IMPWitO/fYBsK+IdBWRApzZzZ8LPEBEuvkX4PRNGFgAbEp7pH5HHw3Ll1tluzHpEM+4ca+3bJznZvZsZw4aEed29uzw88wEztPSuXPdeWpCze/in49m3Li614knxuB4Et0C59tJt7KyMj322GMVp/tCjzrqKP3666+9CyhLkOQ8N4luwCjgc2A1cJ1v34XAhb6frwZWAB8B7wCHxnLelL3PfP6580cwdWpqzm9MFov3fcbzhCWeLVOTm1AJSqrPGW4ivlBbcXH0ZCneeMJNFhht808U6JWamhqdPXu2tmzZUgE955xzvA0oC3iV3KRqS+n7TI8eqsOGpe78xmSpeN9nrFsqSYms0xSLSKOZIL6uomQLlEPFM2VKfCOq/Lweji0ijBkzhpUrV3Leeedxxx137HqsJlJxkTFuOPFEeOMN54/SGJMyltwkKZZ1mlIhVL2OhBo/EkEyXf/BBbzFxc7mnzdn3LjQ9USjRmXGmk9t2rRhxowZtGrl1JdWVFQwZMgQJk2aRHV1tTdBmex3wglOIdoLL3gdiTFZzZKbJIVLEGJJHJKZnTjURHwXXphcgXK8sQS25vzwg7P5W3buvbdufGedBbNmud/K5Ybnn3+ed999lz//+c8ceuihfPJJg5ss26TDgAHQsSM88YTXkRiT3eLpw/J6y8Sam3CFvdEKZ2MpGk5EIgXKqYolWKK/q3R57rnntF27dgpoQUGB3nrrrVpZWel1WBkPq7mJz1/+opqXp7phQ2qvY0wWifd9xlpukhRtnaZwUtWdFVwbE3gt/2R8wcstxBtLoi1OybRypcNxxx3HihUrOO+886ioqODaa6/l4IMPZtmyZV6HZrLJmWdCVRXMnet1JMZkLUtukhRtnaZw0vFBH1jsDE5Xvz/xCowvnliSKaB2Y3LBVNtzzz2ZMWMGr7zyCp07d2bp0qUsXrzY67BMNunVC/r1g0ce8ToSY7KWJTcuiDayKZR0fNDH2iITKZbgVpo//jHxFqdorVxur5CejBEjRrB8+XKmTp3KuefuXpJok41yMW4480z44AP49FOvIzEmK1ly45FEu7PiEWuLTLhYRo2q20oT7rM9lhanwFYucLrJ/InRRRelZkh9MvbYYw8uvfRSfJPc8sUXX9C1a1euueYaduzY4V1gpv47/XTIy4P77vM6EmOykiU3Hkm0Oyse4VpkWraMLZYXX6zbShPvtYKNGbM7mfKPuF6zBqZP92ZIfTz++9//sn37dm6//Xb69evH22+/7XVIpr7aay84+WR46CH4+WevozEm61hy46FEurMiCe7WGTUKCgrqHrdtW90WkVCxxFr/E2+LU6juMg2zzFOmFBsDjB07lrfffpsDDjiAzz77jEMPPZTLL7+cn+3DySTi4oth61Z49FGvIzEm63ia3IjIniLypIh8KiKfiMghXsZTn4Uq9J05Eyoq6h5bWRlbi0ik1hj/hIGJtDjFk7BkUrExwODBg1m6dCnXXnstOTk5TJkyhT59+vDee+95HZqpb4YMgT59YNq08Nm9MSYhXrfcTAFeUtXuQF+g3s+c5lVRbKjWkFCJjV+4kVDBLT/hllhQdRKbRFqcwiUswTMsu12D5JbCwkImTJjA+++/T58+fSgrK6NZs2Zeh2XqGxG49FJYtgxee83raIzJKp4lNyLSDPg/YCaAqlao6hav4nFDqtaZikW83TfBCUao2GfNcmYVduuafuEKmC+8MLU1SG4nnv379+eDDz5g4cKFHHDAAYAzKeb777+fdKymgTjzTGjfPjOzeGPqs3hm/HNzA/oB7wMPAx8CM4AmIY4bCywGFnfq1Mn1WQ/d5OUMvOGuHWoLNftwuFW+O3dOzesaN041N9c5T26ucz+V0jUL8+zZsxXQs846Szdt2uTuyTMUNkNxciZPdv5Dvvlmeq9rTD0S7/uMl91SeUB/4F+qeiDwM3BN8EGqer+qDlTVgSUlJemOMS5ezsAbqjUklNzcui0iF10Ufoj3mjXOlmyXUWCrSatWMGPG7tFS1dVOK1EqW7jStcDptm3bKCwsZNasWfTs2ZNnnnnG3QuY7HP++VBSArfc4nUkxmQNL5ObMqBMVf2VmE/iJDv10pw5zgd3KOkoig21Snd+fu1jGjd2kojAxGbOHGcYdjSqoYuIY+nqCe7y2rTJKWoOlOph3+lKPMeNG8eyZcsYMmQI3333HSeeeCKnnXYaGzdudPdCJns0bgxXXQUvvwz/+5/X0RiTFTxLblT1O2CdiOzv2zUcWOlVPMnwf3j7WyICpbMoNniV7oceqr0ieFFR3edcd13sAzXUV0Q8YYLzPBGnZCBajVGoVpNQUtnClc6lH/bbbz8WLVrE1KlTady4MXPnzqVHjx589NFH7l/MZIdLLnH+M155pfMHbIxJitejpS4F5ojIxzg1OLd6HE9Cwn14B3YBeTWKqrx898+bNsG55zrdQv44/OtOxWrNGucc/ucFJ0ahWmBiTVpS2cKVjhmhA+Xk5HDppZdSWlrKsGHDaNWqFd27d0/NxUz9V1Tk/GdcuhQee8zraIyp/+Ip0PF6S3uhX4xEQhfcijiPp6uYNVg8Rcaxbjk50Y/xv+544kjH72P2bCcWEec21dfzq6mp0fXr1++6v3nzZn3kkUe0pqYmPQGkGFZQ7I7qatX+/VU7dFDdts2bGIzJUPG+z3jdcpMVonV5pKuYNZjb3TyNG8fWYh78+wjValJQsLvLLHCNqVS2aPmXfujUyfndpPp6fiLCXnvttev+FVdcwZlnnskxxxzD2kyagtl4KycH7rkHvvkG/vY3r6Mxpl6z5MYF0bo83Cxmjad7y81uHhGniy2aUF09odauevBBmDKl7hpTqZwXyMt5iAIdccQRtGjRgpdffplevXpx3333UWN1FgbgkENg3Dj45z+dVcONMYmJp5nH6y1Tu6VUI3d5uDVPTLzdW6GOT2abPTv8fDj+1xNPV0+65wXych6iYOvXr9cTTzxRAQX0iCOO0NWrV6c/EBdg3VLu2rJFtV071T59VHfs8DYWYzJEvO8znr+RxLN5/qaToEhJRnGxM4FdLLUgiXw4R0tI4tn8seXn196fn59Y/Uq0WiW3pft60dTU1Ojjjz+uJSUlCmjjxo21rKzMm2CS4FVyA4wEPgO+AK4J8fgY4GPf9jbQN5bzZsT7zPPPO/85//QnryMxJiNYcpOh4kkywrXGJPrhHE9hcUFB+McCC6TDJWPxFO025JabQBs2bNDTTjtNzz33XG8DSZAXyQ2QC6wG9gYKgGVAj6BjfgW08P18DPBeLOfOmPeZSy5x/oMuWOB1JMZ4zpKbDBZPkhHqAzfRD+dwSVFw4uJPRiItxRCJG91mqRw15dWotVhVVlbu+vmtt97SiRMn1tqXqTxKbg4BXg64/1fgrxGObwF8E8u5M+Z95pdfVHv1Um3dWnXdOq+jMcZT8b7PWEFxGsVTQBzq2ETnaolWWNy5szMKyr/Ct7/QN97rxDsqLFShsduLZXp5vXjl5eUBUFVVxXnnncc111zD4MGDKS0t9TiyjNQeWBdwv8y3L5zzgAXhHhSRsSKyWEQWZ8xs0kVF8PjjzmRVJ5xQe9IqY0xk8WRCXm8Z840qQcm23KgmNldLpJqfcC0XiVwn02pa6rOXXnpJO3bsqIDm5+frTTfdpDt37vQ6rJDwpuXmFGBGwP0zgX+GOfYI4BOgOJZzZ9z7zHPPOX9Ep5+umiVzIxkTr3jfZ6zlJo0mTKi73lMokVpJApdY8Le0RBPYYhEoN9cZeXrddXWHlke7Tqgh6U2ahL5+y5bRYzS1HX300Sxfvpxx48ZRWVnJTTfdxKBBg1iyZInXoWWKMqBjwP0OwLfBB4lIH2AGMFpVwywPm+GOO85ZVPPRR23+G2NiFU8m5PWWcd+oEhBc0xLPaCn/86MdG3hMcbGz+X+OVDAcaw1KqJagSOctLk7412VU9X//+5/uvffeCmibNm20vLzc65BqwZuWmzzgS6AruwuKewYd0wlnJNWv4jl3Rr7P1NSonn++8wd1991eR2NM2sX7PuN5whLPlpFvOmkUS0GsG3PbRCscjndZh3DdUl4th1Afbd++Xf/0pz/po48+6nUodXiR3DiXZRTwOc6oqet8+y4ELvT9PAP4EfjIt8UUZ8a+z1RVqZ58svNH9cADXkdjTFrF+z4jznPqh4EDB+rixYu9DsMz4Ra67NzZ6TqKdEw8RCIvs5CT46QtsQqMz88/W3BgAXLjxplV4JvpJk6cyIYNG7jllltoHFwBnkYiskRVB3oWgMsy+n1m504YPRpeftmZxfiSS7yOyJi0iPd9xmpu6pFYlnFwY6miaKOr4lnWQSR0/ZBX621li02bNnHzzTczadIk+vTpw+uvv+51SCYdGjWCZ55xEpxLL4WJE72OyJiMZMlNPRJtgc5Ix8QqliHfoYakhyICF14YuiXGzfW2GqLi4mIWLVpEnz59WL16NUcccQQXXXQRP/30k9ehmVQrLIQnnoDTT4e//tVpvamq8joqYzKKJTf1SCzz3MSaeASKd96XMWPgrLOin/ORR+Dee0M/HkuiZiIbMGAAH3zwAePHjyc/P59//etf9OrVi5dfftnr0Eyq5efDv/8NV14J06bBr38NW7Z4HZUxGcOSm3pizpzdXTm5uc6+UMlI8ER1xcXOFo6/HiaeoeUAL74Yer8IzJ4d/VyJTkhoaisoKOCGG25gyZIlDBw4kLVr1zJx4kTqUy2dSVBuLtx5J8yYAf/9Lxx8MHz8sddRGZMRPE9uRCRXRD4UkefTfe1Qc7VkIn/xrb9QuLp6dyIQKoEInKPmhx+cbfZsd5OJcN1HqvHPvROt1ai+/Dt5qXfv3rzzzjvccccdzJgxAxEBYMeOHR5HZlLuvPNg4ULYtg0OOgimT4+v4t+YbBTP0KpUbMCfgUeB56Md6+YQzUxfZyiQWws+ujn0Ol2LUNanf6dMU1NTo8OHD9fTTjtNN27cmLLr4NFQ8FRtGTsUPJrvv1c9+mjnj+Skk1S/+87riIxxTbzvM5623IhIB+DXOPNRpFV9Gq3jVvFtIrMbh5OubqX69O+UaT755BPeeecdHnvsMXr06MHjjz9u3VXZrHVrp7/4jjvg+eehRw+nydb+zU0D5HW31GTgL0DYWVVStaBdfRqt40XxbbSuoFi6ldzoTqpP/06ZpkePHnz88cccfvjhbNy4kVNPPZWTTz6Z7777zuvQTKrk5MBVV8FHH8H++8OZZzrFxqtWeR2ZMekVTzOPmxtwLHCv7+fDSXO3VLq6VcKJp4so0a6ZRLuhYp0JOdK53epO8vrfKRtUV1fr9OnTtWnTpgpoixYtdM6cOa6dH+uWykxVVaqTJ6vusYdqfr7qlVeqbtnidVTGJCTe9xkvk5vbcBa/+xr4DvgFmB3pOdlSc5PIteNNVMJdI5Z1rKIlFLHE72adkNXcuGPNmjV69NFHK6Djx4937byW3GS4b79VPfdc54++pER16lTVDFufzJho6k1yUysID1puVL1b2ygdrRHhriESPVEIPiZ4jahY4o92jnjYGlTuqamp0SeeeEJ37ty5a9/q1au1pqYm4XNaclNPLF6sethhzh9i+/aq99yjumOH11EZE5N432cyYm0pETkcuFJVj410XEav+RKHcGszRVvTyY1rhBK89lO0NaxiiT+WdbDqs8rKSsrKyur9UOvq6mq+/fZbCgoKKC4uJi8vL+yxhYWFdOjQgfz8/Fr7bW2pekTVmRPnxhvhrbegQwe4/HL4wx+geXOvozMmrHjfZ8K/k6WRqr4OvO5xGGnTqVPoD/5EC4T9E/yto3LxdwAAFX5JREFUXeucY8KE8NcIxV+c6z/PmjVOohKYwASOhIol/gkTQi+MmS2T9JWVldG0aVO6dOmya06Z+mj79u1UVVVRVVVFZWUlrVu3pnXr1nVek6qyadMmysrK6Nq1q0fRmqSJwPDhMGyYMzfOLbc4sxzfdJMzX85ll8Hee3sdpTFJ83q0VIPk5jDqwAn+VJ3bsWNh1KjYl2Ho1KnuRIGqzvsg1B0JFUv88UzSVx/t2LGD4uLiep3YAOyxxx707NmTFi1aUFNTw7p16/jss8/qtEiJCMXFxfW+pcr4iMCRR8Lrr8OSJXDCCc4yDvvu67x5zJsHFRVeR2lMwiy58YCbH/zh5oG5//7aSzWE409KQp1HdXc3UqQlHsLF7+a8Opmovic2fvn5+eyzzz7ss88+5Ofns337dr4O0XeYLa/XBOnf31kI7uuv4dprnSUcfvMbp8vqyithxQqvIzQmbpbceMStD/5w871UV+++DfeZlJu7OymJdz4ZNxMXW14hM7Ro0YKePXvSqlUrOtkKpg1P+/Zw881O8+0LL8DQoTBlCvTqBb17O499/rnXURoTE0tu6rlYPoMCu5j8GjeGWbN2JyVerdIdrlvNEhxv5OXl0aVLFxrHu7S8yR65uU7X1FNPwTffwD//CXvuCTfc4EwMeOCBTnPvxx/b7McmY1lyU8+Fqn8Jxd/FFK4byatVum15BWMyWOvWcMkl8OabsG4dTJoEhYVw/fXQt6/z7WfcOGe5h+A/ZGM8ZMlNPRdc/xKuxsZfOxOqG8k/SiqwRiddBcC2vEJmeOmll9h///3p1q0bEydO9Dock4n8w8bfeQe+/RZmzoRBg5x6neOOg+Jip0j51lvh3XehqsrriE0DZslNPRDLOk/+xGXWrPhaYIJHSVVX7z4+HQXAXnWHpVtFaSnbJk9m6/jxbJs8mYrSUq9D2qW6upqLL76YBQsWsHLlSh577DFWrlzpdVgmk7VtC+ee64yq2rQJXnkFLrgANmxwvikdcgi0aOGsa3X33fDeezb6yqSVJTcZLt6alHhHYrnVLZRoUbBX3WHpVFFaSvn8+ejWrQDo1q2Uz5+fdIJTWlrKkCFDdt1funQpw4YNi/s877//Pt26dWPvvfemoKCA3/3udzz77LNJxWYakEaNYMQImDzZqcPZsAEef9xZtHP1amfE1eDB0KwZ/OpXcMUV8OSTTj2PMSmSEZP4mfAiJR/hEpYxY2JvdXGjW8ifgPnj9Cdg/lgi8T8ePAlhNg0b37FwIVRW1t5ZWcmOhQsp6N074fP27NmT1atXU11dTW5uLldccQV33313rWOGDh3KTz/9VOe5d911F0ceeSQA33zzDR07dtz1WIcOHXjvvfcSjss0cCUlcMopzgawfr3TleXfpk2Df/zDeaxDB2co+oEH7t46dgw/xNOYGFlyk+FSXZPSsqXTqhxqf6wSScACRUrGQs2+XN8SH3+LTaz7Y5WTk0PPnj1ZsWIFq1atolOnTvTv37/WMW+++Wb0+EKMeKkPc9qIyEhgCpALzFDViUGPdwceAvoD16nqXemP0tC2LZx0krOB0z310UdOovPee/DhhzB//u6RV8XF0K+fk+j07Qs9ekD37rHPSmoMltxkPLeXakiFVCVgybQIZRJp3jxkIiMurOUzePBg3nrrLe69915eeumlOo/H0nLToUMH1q1bt+uxsrIy2rVrl3RsqSQiucA0YARQBnwgIs+pamCx0GbgMuAED0I04RQUwEEHOZvfzz87XVoffrh7mzp1d52OCHTt6iQ6gdsBB8Aee3jzOkxGs+Qmw8W6RlOiLRybN8e3P5RUJWDJtghlisLhwymfP79211R+PoXDhyd97sGDB3P22Wdz8cUX0759+zqPx9JyM2jQIFatWsVXX31F+/btmTt3Lo8++mjSsaXYQcAXqvolgIjMBUYDu5IbVd0AbBCRX3sToolZkyZOEfIhh+zeV1kJq1bBypW1t1deqV2c3K4ddOtWd9tnH6fOxzRIltxkuFhqUpJp4XAjMUnVIpnZMkzcX1ezY+FCdOtWpHlzCocPT6rexq979+40atSIq6++OuFz5OXlcc8993D00UdTXV3NueeeS8+ePZOOLcXaA+sC7pcBB3sUi0mF/PzdLTSBqqqcQmV/svPFF8724ovw3Xe1j23d2kl09t7bGV3h3zp1cjbr6spaltzUA9EKhJNp4XAjMUlVUXB96JKLVUHv3q4kM8GmTJnCbbfdRpMmTZI6z6hRoxg1apRLUaVFqKKghKfLFZGxwFjAlp7IdHl5zkzJ++8PJ55Y+7Ht253Ex5/w+Lc33nDm5vGvS+NXUrI72fEnPh07Oq1B7drBXns53Wim3rHkJgsk08LhVmISzwitWKWqRSgbrF69ml//+tcMGTKEs846y+twvFAGdAy43wH4NtGTqer9wP0AAwcOtDUF6qs99nCKkPv2rftYVZWT4KxZ42xr1+7++ZNP4KWXQs+yXFKyO9nxb23b1r5fUmJJUIbxLLkRkY7Av4G9gBrgflWd4lU89VmyLRypSEzc0BCGiSdqn3324dNPP/U6DC99AOwrIl2Bb4DfAad7G5LJaHl5u7ujhg6t+7iqU2y4dq0zfP3bb+tuH30E33/vzJgabM89nW6w4K2kpO6+li2dScFMynjZclMFXKGqS0WkKbBERF4NGu1gYpDNLRyZmngZb6lqlYhcAryMMxT8QVVdISIX+h6fLiJ7AYuBZkCNiFwO9FDVbZ4FbjKXiDMMvbjYGYYeTlWVM1FhYNKzYYOzbdzo3H76KSxa5MyzEWpx0ZwcaNXK2Vq2dK7ZsuXuLfB+4M9NmtgcQDHyLLlR1fXAet/PP4nIJzhFgpbcxMlaOExDpKovAi8G7Zse8PN3ON1VxrgnL293d1Q0VVVOa5A/+QlMgr7/3kl+Nm+Gr76CJUuc++Xl4c9XUFA7CWrRwmkxat689ha8z3+/qKjBJEcZUXMjIl2AA4E606JaoV9srIXDGGMyTF7e7q6oWJWXw48/7k58Nm+u/XPg/TVrnPmBtm6FbdtCd5cFxxMuAWre3KlZato0tttGjTI6UfI8uRGRPYCngMtDNRdboZ8xxpgGo6jI2eKdSFPVGS22dSts2eLchtqCH1u9evfP27fXHVEWTl5e9CSoaVOnKy3S1qMHFBbG/3uKFp7rZ4yDiOTjJDZzVHWel7EYY4wx9ZbI7oSiQ4K9saqwcyf89JOT6CRyu2lT7fuRutnAGanWvXti8Ubg5WgpAWYCn6jqP7yKI1tkwxpMxhhjPCTitKIUFjqjvNxQXe2Mdvn559Bbx47Rz5EAL1tuhgBnAqUi8pFv37W+IkETh2xZg8kYY0yWyc3d3aKURl6Olvp/hJ5l1MQpW9ZgMsYYY9xgswhlgWxZg8l459xzz6V169b06tXL61CMMSZpltxkgXAj5G3kfOaYMwe6dHHm7urSxbmfSc4++2xeeuklr8MwxhhXWHKTBSZMqLu4bbbMUJwN/DVRa9Y4gxH8NVHJJjilpaUMGTJk1/2lS5cybNiwhM71f//3f7Rs2TK5gIwxJkN4Ps+NSZ7NUJzZUlUT1bNnT1avXk11dTW5ublcccUV3H333bWOGTp0KD/99FOd5951110ceeSRiV/cGGMymCU3WcJmKM5cqaqJysnJoWfPnqxYsYJVq1bRqVMn+vfvX+uYN998M7mLGGNMPWTJjTEpluyq7ZEMHjyYt956i3vvvTdkzYy13BhjGiJLboxJsVSu2j548GDOPvtsLr74Ytq3b1/ncWu5McY0RFZQnMEyfYSNic2YMXD//dC5szMBaOfOzn03uhG7d+9Oo0aNuPrqq5M6z2mnncYhhxzCZ599RocOHZg5c2bywRljjEes5SZD2azD2SVVNVFTpkzhtttuo0mTJkmd57HHHnMpImOM8Z613GSoSCNsjFm9ejXdu3envLycs846y+twjDEmo1jLTYayWYdNJPvssw+ffvqp12EYY0xGspabDGWzDhtjjDGJseQmQ9msw8YYY0xiLLnJUKkcYWPcoapeh5BWDe31GmPqL6u5yWA263DmKiwsZNOmTRQXFyMiXoeTcqrKpk2bKCws9DoUY4yJytPkRkRGAlOAXGCGqk70Mh5jYtWhQwfKysrYuHGj16GkTWFhIR06dPA6DGOMicqz5EZEcoFpwAigDPhARJ5T1ZVexWRMrPLz8+natavXYRhjjAnBy5qbg4AvVPVLVa0A5gKjPYzHGFOPiMhIEflMRL4QkWtCPC4iMtX3+Mci0j/UeYwx2cfL5KY9sC7gfplvnzHGRBTQ8nsM0AM4TUR6BB12DLCvbxsL/CutQRpjPONlchOqCrPOcAwRGSsii0VkcUOqbzDGRBRLy+9o4N/qeBfYU0Ta/v/27j1GrrIO4/j3SUuhreVaMJUS25qq8RLaCtgCNo0IxEqoGhMaQkDFeK2XIiFF/rDeIl5jjAmIbY1iKWi52BCUGkEbUXoBtjfacrEVW6DFoLWUpLbl5x/vu3VYd3Z3YGbOmTPPJ5nMmTNzzjzv7Nl3333fM+dtd1Aza78iTyjeCZxW83g88HTfF0XETcBNAJKek/S39sQ7Yizwjza/Z9G6sczQneVuRplf34wgDeqv5/edQ3jNqcAzfXcm6eOk3h2AFyRtG0KGTjlenLO5nLN5GsnYUD1TZONmLTBZ0kRgFzAXuHSgDSLi5HYEqyVpXUSc0e73LVI3lhm6s9wdXOah9PwOqXcYXv5P1JADdMhn55zN5ZzN08qMhTVuIuKQpHnAvaSvgi+JiM1F5TGzjjKUnt8h9Q6bWfUUep2biLgHuKfIDGbWkYbS87sCmCfpVtKQ1d6I+L8hKTOrHl+heHANdVVXRDeWGbqz3B1Z5no9v5I+mZ+/kfSP02zgCeBF4CNNjtEpn51zNpdzNk/LMsrzxZiZmVmVeOJMMzMzqxQ3bszMzKxS3LgZwGCXd68iSTskbZTUI2ld0XlaQdISSXskbapZd6Kk30l6PN+fUGTGVqhT7oWSduWfd4+k2UVm7BRF1g2STpN0v6QtkjZL+nxeX/cYlnRtzrpN0oU169+Rf9+fyFNVNH2Ke0nDJD0i6e6y5pR0vKTlkrbmz3VG2XJKmp9/3pskLZN0TFkyNlqnNppN0tGSbsvrV0uaMGioiPCtnxvpJMUngUnACGA98Jaic7Wh3DuAsUXnaHEZZwLTgE01674NLMjLC4BvFZ2zTeVeCFxddLZOuhVdNwDjgGl5eQzwGGkKin6P4fzceuBoYGLOPiw/twaYQbom0G+A97Yg71XALcDd+XHpcgI/Az6Wl0cAx5cpJ+nik9uBkfnxL4EPlyVjI3XqK8kGfBq4MS/PBW4bLJN7burzxJ4VFRGrgOf7rJ5DquDI9+9va6g2qFNua1yhdUNEPBMRD+flfcAW0h+/esfwHODWiDgQEdtJ3x47S2kqimMj4i+R/mr8nCYf95LGA+8DFtWsLlVOSceS/jgvBoiI/0TEv8qWk/Tt5pGShgOjSNdsKkXGBuvUV5Ktdl/LgfMG63Fy46a+bp3YM4CVkh5SuiR9t3ht5Gug5PtTCs7TTvOUZs1eUsXhuBYoTd2Qu+enAqupfwzXy3tqXu67vpl+AFwDvFSzrmw5JwHPAT/Nw2eLJI0uU86I2AV8F3iKNH3I3ohYWaaM/WhmtiPbRMQhYC9w0kBv7sZNfUO+dHvFnBMR00gzKn9G0syiA1lL3QC8AZhCqjS/V2ycjlCKukHSa4DbgS9ExL8Hemk/62KA9U0h6SJgT0Q8NNRN6uRp9ec9nDSkckNETAX2k4ZR6ml7zvxPxxzSMM7rgNGSLhtokzpZynDsvpJsDed246a+rrx0e0Q8ne/3AHeSuuC7we7cLUq+31NwnraIiN0RcTgiXgJ+Qvf8vF+NwusGSUeRGjZLI+KOvLreMVwv78683Hd9s5wDXCxpB2no7t2SflHCnDuBnRGxOj9eTmrslCnne4DtEfFcRBwE7gDOLlnGvpqZ7cg2eVjuOAYZYnfjpr4jl3eXNIJ0EtOKgjO1lKTRksb0LgMXAJsG3qoyVgBX5OUrgF8XmKVteiuf7AN0z8/71Si0bsjnGiwGtkTE92ueqncMrwDm5m+cTAQmA2vyUME+SdPzPi+nicd9RFwbEeMjYgLpM7ovIi4rYc5ngb9LelNedR7waMlyPgVMlzQq7/s80rlWZcrYVzOz1e7rQ6RjaeAep1d7lnSVb6RLtz9GOpv7uqLztKG8k0hnsa8HNle1zMAy0hDMQdJ/BFeSxm9/Dzye708sOmebyn0zsBHYkCuQcUXn7IRbkXUDcC6pS34D0JNvswc6hoHrctZt1Hw7BjiD1KB9EvgR+ar1Lcg8i/99W6p0OUnDsuvyZ3oXcELZcgJfAbbm/d9M+rZRKTI2Wqc2mg04BvgV6eTjNcCkwTJ5+gUzMzOrFA9LmZmZWaW4cWNmZmaV4saNmZmZVYobN2ZmZlYpbtyYmZlZpbhxY2ZmpSDphXw/QdKlTd73l/o8/nMz92/l4saNmZmVzQSgocaNpGGDvORljZuIOLvBTNZB3LixtpB0v6Tz8/LXJf2w6ExmVlrXA++S1CNpvqRhkr4jaW2e5PUTAJJm5brlFtLFKJF0V574d3Pv5L+SrifNqN0jaWle19tLpLzvTZI2SrqkZt9/kLRc0lZJSwebidrKY3jRAaxrfBn4qqRTSLMYX1xwHjMrrwXA1RFxEUBupOyNiDMlHQ08IGllfu1ZwNsiYnt+/NGIeF7SSGCtpNsjYoGkeRExpZ/3+iDpCsWnA2PzNqvyc1OBt5LmOHqANF/Wn5pfXGs299xYW0TEKtLMrlcBcyPisKRJkhZLWl5wPDMrtwuAyyX1AKtJl/afnJ9bU9OwAficpPXAg6TJFiczsHOBZZEmkN0N/BE4s2bfOyNNLNtDGi6zDuDGjbWFpLcD44ADEbEPICL+GhFXFpvMzDqAgM9GxJR8mxgRvT03+4+8SJpFmkF7RkScDjxCmpdosH3Xc6Bm+TAe7egYbtxYy+WZp5cCc4D9ki4sOJKZlds+YEzN43uBT0k6CkDSGyWN7me744B/RsSLkt4MTK957mDv9n2sAi7J5/WcDMwkTc5oHcyNG2spSaOAO4AvRsQW4GvAwkJDmVnZbQAOSVovaT6wCHgUeFjSJuDH9N+L8ltguKQNpLrmwZrnbgI29J5QXOPO/H7rgfuAayLi2aaWxtrOs4JbYSSdBHwDOB9YFBHfLDiSmZlVgBs3ZmZmVikeljIzM7NKcePGzMzMKsWNGzMzM6sUN27MzMysUty4MTMzs0px48bMzMwqxY0bMzMzqxQ3bszMzKxS/gtVAwa2oStLzQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(8,4))\n",
    "\n",
    "x1 = np.array([X[:,0].min()-1, X[:,0].max()+1])\n",
    "x2 = - w.item(0) / w.item(2) + x1 * (- w.item(1) / w.item(2))\n",
    "\n",
    "# Plot decision boundary？？？\n",
    "ax[0].plot(x1, x2, color='k', ls='--', lw=2)\n",
    "\n",
    "ax[0].scatter(X[:int(n_pts),0], X[:int(n_pts),1], color='lightcoral', label='$y = 0$')\n",
    "ax[0].scatter(X[int(n_pts):,0], X[int(n_pts):,1], color='blue', label='$y = 1$')\n",
    "ax[0].set_title('$x_1$ vs. $x_2$')\n",
    "ax[0].set_xlabel('$x_1$')\n",
    "ax[0].set_ylabel('$x_2$')\n",
    "ax[0].legend(loc=4)\n",
    "\n",
    "ax[1].plot(loss_history, color='r')\n",
    "ax[1].set_ylim(0,ax[1].get_ylim()[1])\n",
    "ax[1].set_title(r'$J(w)$ vs. Iteration')\n",
    "ax[1].set_xlabel('Iteration')\n",
    "ax[1].set_ylabel(r'$J(w)$')\n",
    "\n",
    "fig.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. 预测的准确性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "预测的准确度是: 98 % \n"
     ]
    }
   ],
   "source": [
    "X_1 = np.hstack((np.ones((X.shape[0], 1), dtype=X.dtype),X))  #添加一列特征1\n",
    "y_predictions = sigmoid(X_1 @ w)>=0.5\n",
    "\n",
    "print ('预测的准确度是: %d ' % float((np.dot(y, y_predictions) \n",
    "                               + np.dot(1 - y,1 - y_predictions)) / float(y.size) * 100) +'% ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 6. Scikit-Learn库的逻辑回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scikit score:  0.98\n",
      "预测的准确度是: 98 % \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2554ab76850>]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAREAAAEYCAYAAAB7m8JGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2deZwU5bX3v2c2GFQwjLi8yoAa4zq4YaIRE5FgXAGTmKvXBYwRNeorCvHGSxSN4nXD9UoURZYXRM0VXK7iRlwwagyoMPBB1EREokZEAZGBmWHO+0d1Dz09Xd3VXVVd1TPn+/nUZ6ZrPQXz/Po85znPeURVMQzDKJSyqA0wDKO0MRExDMMXJiKGYfjCRMQwDF+YiBiG4QsTEcMwfGEiYuSNiFwjIjOitsOIByYiJYSIDBCR10VknYh8JSJ/EZHDorYrH0RkhYg0iMg3IrI28T4XiIinv0UR6SsiKiIVIdtZlOd0BExESgQR6Q78L3A30BPYFbgW2BylXQVysqpuB/QBbgT+A5gcrUlGoZiIlA7fA1DVWaq6RVUbVPV5VV0MICJ7isifRWSNiHwpIjNFZPvkxQkP4LcislhEvhWRySKyk4jMTXgFL4rIdxLnJr+FR4rIpyLymYiMdjNMRA5PeBRrRWSRiBzt5YVUdZ2qPgn8GzBcRA5I3O9EEXlHRNaLyCcick3KZa8mfq4VkQ0icoSHd/8PEfln4j2Xi8igxP4yEfmdiPw9ce2jItLT7Tle3qlToqq2lcAGdAfWANOA44HvpB3/LjAY6AL0wmkEd6QcXwG8CeyE48V8AbwNHJy45s/AuMS5fQEFZgHbAHXAauAniePXADMSv++asOsEnC+lwYnPvVzeY0XyPmn7VwIXJn4/OvHMMqAf8C9gWJptFV7eHdgb+AT4PynX75n4fVTi32S3xLX3AbPcnmNb5s08kRJBVdcDA3D+sO8HVovIkyKyU+L4h6r6gqpuVtXVwG3Aj9Nuc7eq/ktV/wnMB/6qqu+o6mZgDo6gpHKtqn6rqvXAFOD0DKadCTyjqs+oaouqvgAswBGVfPgUp5uGqr6sqvWJ+y3GEbP0d2klx7tvwRGI/USkUlVXqOrfE8fOB8aq6qrEv8E1wC8sDpIfJiIlhKouU9URqrobcADwf4A7AERkRxF5OOG2rwdmADuk3eJfKb83ZPi8bdr5n6T8/nHieen0AU5NdGXWishaHLHbJc/X2xX4KvEuPxCRl0RktYisAy7I8C6tZHt3Vf0Qx+O4BvgicV7yPfoAc1LsXoYjOjvlaXunxkSkRFHV94CpOGIC8F84Xko/Ve2O4yGIz8f0Tvm9FsdbSOcT4P+p6vYp2zaqeqPXhyRGmHYFXkvsegh4Euitqj2Ae9n6LpmmnWd9d1V9SFUH4IiGAjel2H58mu1dE56aTW/3iIlIiSAi+4jIaBHZLfG5N0734s3EKdsBG3ACgbsCvw3gsVeJSDcR2R84B3gkwzkzgJNF5KciUi4iXUXk6KSdOd6pu4icBDyME2OpT3mXr1R1k4h8H/j3lMtWAy3AHin7XN9dRPYWkWNEpAuwCcfj2pI4fC8wXkT6JM7tJSJDszzHyICJSOnwDfAD4K8i8i2OeCwBkqMm1wKHAOuAp4HZATzzFeBDYB5wq6o+n36Cqn4CDAX+E6fhfYLTiLP9bT0lIt8kzh2LE8M4J+X4b4A/JM65Gng05XkbgfHAXxLdkMPJ/u5dcIaRvwQ+B3ZM2ApwJ47H83ziWW/i/Bu7PcfIgCQi0YbRioj0BT4CKlW1OVprjLhjnohhGL4wETEMwxfWnTEMwxfmiRiG4QsTEcMwfFFS6b077LCD9u3bN2ozDKPTsXDhwi9VtVemYyUlIn379mXBggVRm2EYnQ4R+djtmHVnDMPwhYmIYRi+MBExDMMXJiKGYfiipAKrhpFOY309m+bNQ9etQ3r0oOugQVTV1UVtVqfCRMQoWRrr62l46iloagJA161zPoMJSRGx7oxRsmyaN69VQFppanL2G0XDRMQoWXTdurz2G+FgImKULNKjR177jXAwETFKlq6DBkFlZdudlZXOfqNoWGDVKFmSwdNCR2dsZCcYTESMkqaqrq6ghm8jO8Fh3RmjU2IjO8FhImJ0SmxkJzhCFxEReVBEvhCRJWn7L0ksrrxURG4O2w7DSMVGdoKjGDGRqcB/A9OTO0RkIM5aJf1UdbOI7FgEO4wSoRgBz66DBrWJiQA2slMgoYuIqr6aWMcklQuBGxOLKKOqX4Rth1Ea+Al4ZhOfTMcqDzyQpoULQRVEqDzwQAuqFkBUozPfA44SkfE4SxuOUdW/ZTpRREYCIwFqa2uLZ6ERCdkCntkaeDbxAdofe/xx50BytQNVmt55h8ba2lgISSkNP0clIhXAd4DDgcOAR0VkD82wfoWqTgImAfTv39/Wt+jgFBrwdBWfuXPRTZu2ikWSlpb2N9myhU1z50beWEtt+Dmq0ZlVwGx1eAtn4eQdIrLFiBGFBjxdxaehob2AZLtPQ4Pnc8Oi1IafoxKRx4FjAETke0AVzoLLRien0FT2jjSqUmrDz6F3Z0RkFnA0sIOIrALGAQ8CDyaGfRuB4Zm6Mkbno9BU9oyjLYVQXR15PEJ69MgoGHEVymKMzpzucujMsJ9tlCaFpLJnEh9tbIRM3RMRKg89lKa3324bGykro3L//SOPR5Ta8LPNnTE6DOnikx6gBKCykuqTT6aqro7G2tp2Hkeho0NBvwcUPrGw2JiIGB2WXI0xk8fTMHt2xnsVOx5R6MTCKDARMTo0+TbGIOIRUcdUio1NwDOMFPwWOkp2oZJClIypNNbXB21qbDARMYwUqurqqD75ZKS6euvOCu8Oe6nleASBiYhhZECbm7d+aGjw7E2UWo5HEJiIGEYafryJzlhiwAKrRklQzGClH2+i1HI8gsBExIg9RZ+QVl2dOUktNU7iQqnleASBiYgRe4qdACZApjkY4vH6UsrxCIIOGROZOHEiX3/9ddRmGAFR7GCl20zeOMzwjSMdTkQeeeQRLrroIg444ACeffbZqM0xCqSxvp71d9zBumuvBcnsA4QVrOyMwVE/dDgROeSQQzjiiCP49NNPOf7447ngggvYsGFD1GYZeZCesJWxHojPYGWqSK2/4442w7e2sl5+dDgR2WuvvZg/fz433ngjVVVV3HffffTr14/58+dHbZrhkYwxEGj1SKRHj9ZJdIWQK6u0NeEs4Xlkel42EepsSCmV8ejfv78uWLDA8/n19fWcddZZLFq0CBFh9uzZDBs2LEQLjSBYd+21rsd6jBvn+/7r77jDdX5M91Gjcl6fa3ZwR0REFqpq/0zHIlt3JnFsjIioiIRSGrGuro633nqLsWPHss8++3DssceG8RgjYMKOSfgN1HbG1PZsFKM7MxU4Ln2niPQGBgMrw3x4VVUV119/Pe+88w7dunUD4JtvvmHChAk0+a2CZYRC2DEJvyIVt9T2qLtWUa07A3A7cAXwRNg2AHTp0qX199GjR3P//fcza9Yspk+fzn777VcMEwyPpCdsIQJNTTTMns2muXPpevzx7boN+WS0+s4qdUlGK8RT8puJG4fK8JEEVkVkCPBPVV3k4dyRIrJARBasXr06kOeffvrp9OnTh4ULF3LIIYcwYcIEtmzZEsi9jWCoqqtzGnVZWZvRGW1ooOGJJ9p82+Y7/d5L4NSNxvp6aGxsf6CsLG9PaePTT9Mwe7avsgFx6FoVPWNVRLoBYwFPAYow1p0ZOHAgixcv5vLLL2fy5MmMGTOGJ554gqlTp7LHHnsE8QgjADbNm+e+PkxKtmohGa2FZpVumjcPMnzhSJcueXlHjfX1NGUaJMgzEzcOXaso0t73BHYHFokzZLcb8LaIfF9VPy+WEd27d+eBBx5g2LBhnHfeecyfP59+/fqxbNkyevfuXSwzjCxkawipx4JsSLm6F1nXt0m7T7ZuRjZPQdet89zNiUNl+KKLiKrWA60LeIvICqC/qkay7sxJJ53EkiVLuOiii+jSpYsJSIxwayDJY1s/SOaEtAyZrrm8g1zxBa+NNpd3lFXgqqs9xzniMGu4GEO8s4A3gL1FZJWInBv2M/OlpqaGhx9+mAceeKB138KFC5k5cyallEfT0WiNiaRTXt62kbj9H6XtzxU78RJf8DpylMs7yuYpSOK52exI4ie+ExRRrjuTPN43bBu8Upn449i8eTNnnXUWy5YtY86cOfzxj3+kV69eEVvX+Ug2hIa5c7eOhlRWIhUVzkjNvHl0HTTIt3fQMGeOa5V3aCsIXqf657LJbbGtst13p+Wjj3LakUrUs4Y7dMZqoagqkydP5rLLLmPDhg3suOOO3H///QwZMiT0ZxvuuGWKVh54IE2LFuXMIM2WCZsNr5msXmxNtWnj00+3D64mRDLjjOFEty2SVfmizFgtRUSEX//619TX1/PjH/+YL774gqFDh3LOOeewrgPXyow7bp5E8wcfeHLpCwo2Fhhf8NLNaFq6tP2FTU1OLZP0LhO0ds/iVkHePJEctLS0cNddd3HllVeyadMm6urqePfddynL1Fc3QsXvnJqM3kEOKvv3p9uJJ3o+3+256d0fcF8oC6D6Zz9rm2yXoZ0W4iEVSjZPxCqb5aCsrIxRo0Zx3HHHcfbZZ3PxxRebgESE3+HMjJmwOb5EmxYsoLG21veM4fSRFsmyDIX06NEmzuEmnnGpIG8i4pF99tmH119/nfLy8tZ906dPZ6+99uKII46I0LLOQxDDmamN06tn0vD4463X5otbF0yzPDP9feKQC5IN+0rNg4qKChIJcixfvpyRI0cyYMAArrzySjZv3hyxdfEmiEliQQ9npt/PlZaWgtPI8/UWpLo6Yy5InIskmSdSIH379mXUqFHccsst3HjjjTzzzDNMnz6dAw88MGrTYkeQk8SyDWcWMpkteb/G+nrPw7z54JowV13tzMFJTaEvL6fr8cdntBHiW0HeRKRAunTpwo033siQIUM4++yzWbx4MYcddhjXXHMNV1xxBRV5LL0YFcVay6UY1dr9ClVVXR2b5s51LcZcaNfBrQtWuf/+NL39dtuTs8Rnos4FyUb8/9Jjzg9/+EMWLVrEFVdcwcSJExk7diyrVq1i4sSJUZuWlWJOIS/GJLF8hCpVPFNzLyr235+md95pP8GugBm6Sdy8iIyTCxPdpriKhRsWEwmAbbbZhnvuuYfnn3+evffem8suuyxqk3JSzCnkxaie7lWo3IpA67p1NC1aROXBB7ddzLu6muphw7I27ELiPXGYfRsU5okEyODBg1m6dGnrCI6qMm7cOM4991z69OkTsXVtKeYfcTEmiflKfU+SSFzrfsUVnp+by6NzPR5gYaOoMU8kYFKHgKdNm8Z1111HXV0dU6ZMidVkvmKurVKMSWJ+J8Z5PZ5OLo/O7bgk7MtlbylgnkiInHDCCZxyyinMmTOHX/3qV8yZM4dJkyax8847R21aKN5BtkBt2IFBvxPjUo/nQy6PLlv9kdSs1LiNuOSDiUiI7Ljjjjz22GPMnDmTiy++mKeeeooDDjiAe++9l1/84heR2hb0sGEcan16ESq32bNAQSKaqxuV7XicR1zywbozISMinHnmmSxZsoRjjz2WNWvWcOqpp/Lwww9HbRpVdXV0HzWKHuPG0X3UKF9/0HGo9emFdglmPhfEytWNinuiWBCYJ1IkdtttN5599lnuu+8+pk+fzimnnBK1SQWTqdsS19EGty5WUB5ALo8u7oliQRD6LF4ReRA4CfhCVQ9I7LsFOBloBP4OnKOqa3PdK4pZvGHQ0tLSOonvq6++4oYbbmDcuHFst912EVuWG7c6GTQ3u5Yo7HH11cUzMIVs9UeaP/igwzbqMIi6nshU2i9e9QJwgKr2A94HriyCHbEhdRbwqFGjmDBhAv369eOVV16J0CpvuHVbvJYoLCZutjYtWNC2ROLjj7P+5pttXd0CCV1EVPVV4Ku0fc+ranPi45s4Fd87JVdccQUHH3wwK1asYODAgVx++eU0uKRex4G8J5T5GDL2O2nPs60tLa3p7nEr+FMKxCGw+itgrtvBMBavihMHHHAAb775JldffTVlZWXcfvvtHHroocS12+YqCtXVgQYQ812QKi9bcxHDgHCciTSwKiJjgWZgpts5YSxeFTeqqqq49tprOemkkzj77LNZtmwZRxxxBO+//z6777571Oa1wS2/pDox+zRTALGQiX5BTNqr2GuvzAtEeSCbF1OsiYulQmQiIiLDcQKugzROqZwRcthhh/H222/z+9//nsbGxtgJCHgfjUhSaP6I39Gexvp6p3hzgbh5MYW8T0cXnUhERESOA/4D+LGqbozChrhSXV3NhAkT2qTIz58/nzfeeIPRo0e3SauPinyGSAv1KPxW88o6Ryad8vK2M3ezdMPyfZ84JOGFTegikli86mhgBxFZBYzDGY3pAryQqBT2pqpeELYtpUSyglpjYyMjRozgH//4B08++STTpk1jzz33jNg67xTqUfhNy/fksSSWcADv3bB836cYtVSiJqrFqyaH/dyOQlVVFXfffTe//vWv+ctf/kK/fv249dZbueCCC1qFJs4U6lH4TdJynSPjsnaL126YVFdnLFzk9j5xTcILEstYLQFOOOEElixZwsUXX8ysWbP4zW9+w+OPP87kyZPZbbd4j47n8ijCmrTnGgD2mNruWmC5osIZhfLoIcW9yHIQxGGI1/BAz549eeihh3jkkUfo2bMnzz//PAMHDmRLehWumJGtDEAQw7iFPNcLrp5CQ0PW90nPa7G5M0bs+OUvf8lRRx3FeeedxznnnBOLQGsu3DyKoOMFmcoekqHL52W0JN/Zt27dn+qTT6b65JNtdMaIF7vssgtPPfVUm5jIXXfdRW1tLcOGDYvQsvwoJF7gJgDt5skkR7fSlp5sXrmyzbq9bqMl+QZ2swmi3xnScce6MyVKqoC89957jB49mlNOOYXhw4ezdm3OuYyxIN/qatm6P56GdJuaaFq40FPJgny7Q50hgOqGiUgH4Hvf+x633XYbXbt2Zfr06dTV1fHiiy9GbVZO8o0XZPu299xYXfIaM12fT72VYpabjBsmIh2AsrIyLrnkEt59911+8IMfsGrVKgYPHszFF1/Mt99+G7V5rgT5be+5sboMi/tt7J0hgOqGxUQ6EHvvvTevvfYaN998M9dccw333HMP3377LVOmTInatIzkmw6eLdiZtexhkkQtkdSYSHJ/emNvrK+nYe7c1orsUl1N1+OPd7WvMxQfciP0okRB0lGKEhWDd999l0suuYQZM2bEbrkKcC8YlOqJpItMxV57tRcAtjbw5pUrXSfc5TMhsLG+3lnEO31xqfJyqocOjaUwhD0/J1tRIhORTkJLSwsXXXQR559/PgcddFDO82fOhLFjYeVKqK2F8ePhjDOCs2f9HXdkzyitrkY3b27bkJOexNKlGddsccUlSzVv23DEqPuoUd6fXQS8CLJfoq5sZsSAKVOmcO+993LYYYdx/fXX09zc7HruzJkwciR8/LETh/z4Y+fzTNeCDfnjGghNDsk2NLT3BBKLS0lVVZ4PazvMm5rMlilBLFuQNo6jLVEXyTYR6SScfvrpXHLJJTQ3N3PVVVdx5JFH8t5772U8d+xY2Jg2t3rjRmd/UBQayNR16/w15JTG5TZk3GYZzTTiONoS9fCyiUgnoVu3btx11128+OKL9O7dm7feeouDDz6YO++8k5a0b/yVKzPfw21/IXQdNMiZgp8v1dW+G3KycbnOjwEoy9A0ysvbzPnxU7oxSKIeXjYR6WQMGjSI+vp6RowYwaZNmxg1ahTTp09vc05tbeZr3fYXTAHzfgSX4dQ2J0nbn+mHE40r6/yYYcOcko/Ja6qrW4OqYc75KYSoh5dtiLcT0qNHD6ZMmcKwYcOYPHkyZ555Zpvj48c7MZDULk23bs7+IGgNBBaANjRsHU6dO7f9tPyUgKJbwDHZuApdnS5uNUKiHl4uRlGiTOvO9AQeAfoCK4BfqurXYdtitGXo0KEMHTq09fPnn3/OmDFjuOWWW5g0aZfQRmc8pagnJ9Cl7054EclGnquUQPJ5mY4XWvgo6hhEJqJckrMYnshU4L+BVJ/5d8A8Vb1RRH6X+PwfRbDFyMKYMWOYOXMmc+fOZeLEiaxY8W8F3SdXzkLWxpZSbcxLA8/VeLIdzyYy2d6hM9QIyYdiVDZ7VUT6pu0eilMyEWAa8DImIpFz00038eWXX/Lcc89x2mmnMWfOHO655x5qamo838NLTdFsVcfScxvCdtHzmdafPN9v6caORlGSzRIi8r8p3Zm1qrp9yvGvVfU7ue5jyWbho6pMmjSJ0aNH8+2337LzzjvzwAMPcOKJJ3q63i1RKzVJK+zkqEKzN9vUI8lA8l7pNUuyPaOjVHrPlmwW+8CqiIwERgLUBj48YKQjIpx//vn85Cc/YcSIEbz22msMGTKE5cuX893vfjfn9V7iBX4DgdkaZqHV1TMKW4Z3aFezJOGBdNZK7xCdiPxLRHZR1c9EZBfgC7cTO8PiVXFkzz335OWXX+aOO+5g7dq1ngQEvMcLCg0E5mqYhY6ceA72Zrl3urhpY6PvUZxS8GSiyhN5Ehie+H048EREdhhZKC8vZ/To0Vx33XWt+5577jlGjRrlul5w2DkLuVK88x05SSaN5RxZqazMWoskU+6I2/yefBbgilM+ihuhi0hi3Zk3gL1FZJWInAvcCAwWkQ+AwYnPRsxpamri/PPP58477+Tggw/mrbfeaneO3wLJucglEvlkb6Y3UjeS75Dt3vksluVrAa4YrhMcuoio6umquouqVqrqbqo6WVXXqOogVd0r8fOrsO0w/FNZWcljjz3Gfvvtx/Lly/nhD3/IVVddRWNjY5vzkiMYya7NpnnzAvv2zLaguKtH4eIJ5Wz4lZVU/+xnrVXNsnlZnnNEAliAK26TAC3tvRMxcyb07etMC+nbt7BZuYceeigLFy5k9OjRtLS0cP3117Prrj+gvLweEaiogPP/7cvQ3PCMDbmsDBobXWMxbp5QtsYo1dVQUUHD7Nmtc2OyelluC4mJFOyVRT0nxiuxH50xgiE5vT+Zyp6c3g/5Z6J27dqVW2+9lSFDhnDSSSP48st3gWHAcrZsqWDSozVs+ccgJpz07NaLAkoLzzSyQ2Oj66p02Wp/uAaBq6vR5mbX4G2mvBK3eAmqBdcfKZV8FPNEOglhTO//0Y9+xLffLsIZgZ/I1u8kYerC9ikFQbnh6QWUMwmIl+e5dU8UPMcics0D8uM1hB1fCgrzRDoJYU3vb2nZDrgvbe+1bNEaWlpaKEuZUh+WGx70er8Ns2dnPD/TM7LGVQLwGqKcE+MVE5FOQm2t04XJtN8P5eXpM/rfB64FlJ/N2IP/HjqU3Xr08NSgCs2J8OP2Z2qkblmrmUQpm7cTR68hDKw704FJDaRu2NDecw9ien8yrrKV7wGP0qWqJy//4x/8cOJEHn7/fbqedJKnjNFCgrFBu/1u9Uoq9tqr3b5swc/OICBgnkiHJT2QumYNVFVBTQ189VVw0/snTnR+TprkeCTl5TBy5C+4+uoBjBw5kqeeeooLHnqIZzdu5L777mPHHXfMeB+/NTqCdPur6uoyVo5vWrSIxtpaX8ttFkLcs1Y9eyIiMlhE7heRgxKf230HGfEhUyC1sRG23dapf7xiRWYBKWQYeOJEaG52Biiam53PO++8M0888QRTpkyhe/fuPP744/z2t791vUfcciKaP/ig/c4AltvMl1LIWs3HE/kNcA7w+0RRodzrDhiRUUggNchhYHAm840YMYJjjjmG0aNHc9NNN7mfG7MaHfmIWpjBz7hVUctEPjGR1aq6VlXHAMcCh4VkkxEAhdRJDavKe21tLX/605/YeeedAWhububnP/85zz33XOs5UdcJTScuiV5x89AykY+IPJ38RVV/R9tKZUbMGD/eCZymkiuQWowq7wDTpk1j9uzZHHfccVx44YVs2LAhdjkRcRG1uIhZNnIWJRKRO4DLNAZL5VlRovzIdxW7vn0zDwP36ePEUIKiubmZW2+9lauvvpqmpib22GMPpk2bxoABA4J7SADEIaBZjNXtvJCtKBGqmnUDrgeeArolPh8L/CXXdWFshx56qBrhMWOGarduqk6I1Nm6dXP2h8GiRYv0wAMPVEBFRMeMGaMNDQ3hPKyE2bx4sa67/XZde801uu7223Xz4sVFtwFYoG4a4XagzUnw78DfgNeA54CjvFwX9GYiEj4zZqj26aMq4vx0ExCv5+Vi8+bNOnbsWC0rK1NAJ0+eXNiNjFDxJSLAIOAlnGLKy4G9c10T1mYikpmgGrTXe4fhsbz55pt67rnn6pYtW4Iy3QgQvyLyZ2BA4vc64F3gmFzXhbGZiLQnzC6I271ratruS259+vh/ZpKPP/5YBw0apEuXLg3upkbB+O7OtLkAdgFez/c6l3tdBiwFlgCzgK7ZzjcRaU+fPuE1aLd7u20i/p+Z5KyzzlJAu3TpohMmTDAPJWKyiUjec2dU9bNEF8cXIrIr8H+B/uosJVEOnOb3vp2NMIdl871HkMX47777bs455xw2b97M6NGjGThwIB999FFwDzACo6AJeKqauYBD/lQA1SJSAXQDPg3ovp0Gt4bbs2f7ffmmtLvdu6Ym/xyUfOnRowcPPvggTz75JDvttBOvvvoq/fr14/777096sUZccHNRirEBlwIbgNXATJdzRgILgAW1tbVheGolzYwZqpWV7bsWVVVt4yKFxE6yXRNmMDed1atX66mnnqqAVlRU6PLlyzPaWix7OiMEGRMJagO+gxO07QVUAo8DZ2a7xmIimfES6PQSO3EbiUm9f01NdA101qxZetttt7WzNxmPKVZ+S2ckriJyKjA55fPZwMRs15iIZCa9AWUKdOY6x83ruPDC4iageWXGDNWqqscUfqGwOvTRos5ONhGJsijRSuBwEekmIoITrF0WoT1FJ4jq6+Aeuygr23rPXBPy3CbfTZrkb1JecnGoddde21o1PQj+8z+baWwcA/wPcABOUnVbgp7zY2QmMhFR1b/i/AW8DdQnbJkUlT3FJjnt/uOPneWa0PIAABIfSURBVO/N5LT7QoQk02Q7cIoEJe+Za0KeW4NrW/pwK14aaJi1MD75pAKYB/wI+BcwBPgVsL71HFu6uThEWh5RVcep6j6qeoCqnqWqm6O0p5i4ffNfemn+3skZZzgeQ3l5+2NJryF5Tp8+zhIpffo4n5MT8vJtcMnzs3lTYa7g5jx/d5xk6tuALsAUnHzIPwc+WmS4k3MWb5zoSLN4y8rclypJpVu3to29kHuKONXMspFekCgbIs5zampg/fq2OpFq77prr3W9R49x43I/KC97l+GE1RZQUbEnDzywjOHD29dJNQoj2yxeK9QcEV6/+fOJPxRSiChJuqeSyatJkhSqNWvaOxobN8KZZzpeyZ8++H7G64OohdHes9qXqVNf5w9/+APz5j3YKiCl9CVZqpgnEhH5fvOnehKZ6oSA0xVas6bttfl4Mql49ZSy0a1rC3ee/CSn7r9o684i18K49NJL2XbbbRk3bhxVVVVFeWZHxFc9kThtHW2INz0vw0u+R6ah2MpKJ7ks/To/OR35zptx22p32RxZLYwPP/ywtcRAv379dNGiRUV7dkeDOOaJFLJ1NBFJx0tWaT6Nu6am8CzOTLa45ZoUa1JeIcyfP1/32GMPBbSyslJvuOEGbWpqitaoEsREJCTCSLXOdc9CGnKhSWLptmRKPMu1xSHh65tvvtELL7xQAQX08MMPz5g6b7hjIhICxS4lmMRvN8Nvo86Wau4mWnGZ1/Lcc8/prrvuqoCOGDEiGiNKFBOREPBbx6PQhpVPTCRX98Jv487kqRSjCpofvv76a73kkkt0zZo1rftaWlqiMaaEMBEJAS/zVdzw27DcJsrlE6QtVuMOs2hSEGzatEmPOuoonTJliolJFkxEQsBP4yhGw7rwwuwzW/O1oVCvxY/YFoNp06a1xkqGDBmin332WdQmxRITkRDw800edsNyG1m58MLCbPDzrnH3RFpaWnT69Onao0cPBbSmpkb/9Kc/RW1W7DARCYlCv53Dblhe7p+PDX7sjVtMxI2VK1fq4MGDW72S008/vU3cpLNjIhIzwm5YXryMfKqWeQnS5npft8JGcRm5UXW8knvuuUe7deumgE6dOjU6Y2KGiUgMCbPxuDX8mprcNuSTZJbPSFQpFTz64IMP9Morr2wTaO3s1eZNRDoZTtWv9o2+sjJ3A3UTID/lB93uWV7uT5yKxfLly3X//ffXV155JWpTIiObiEQ6i1dEtheR/xGR90RkmYgcEaU9pUp6TQ+A7bZrf15TU+4ZwW7FhlSdqf+ZapHkIoyCR8Xk5ptvZunSpRx99NGMHj2aTZs2RW1SvHBTl2JswDTg14nfq4Dts50fd08kiv59pq5CtsSzXHGMbDGQQrsape6JbN68WX//+99reXm5Arrvvvvq3/72t6jNKirEsTsDdAc+IlGOwMsWZxEplTT49Aaa7/yYQhp4FDGRMAT9r3/9q+6zzz4KaHl5uV599dXa2Njo/8YlQFxF5CDgLWAq8A7wALBNtmviLCJR5UPkMyEvvYFmWrOmstJp3H5HZNJxa9RhTWIMS5w2btyoo0aNUnCW+OwsE/niKiL9gWbgB4nPdwLXZTivJBaviiozMx9PJL0RuaXGJ0sI+BFFL6n5YXlpxRD0l156SR988MHWzy0tLR16BCeuIrIzsCLl81HA09muMU+kPZm+db12Y3Jd4zYik0sM3OI06V5PWN29KAR92rRpOmDAAP373/8e3kMiJJYi4tjFfGDvxO/XALdkOz+OIhKHVdhSG3VNjWpZWVs70od2vQpP6jvlMys3H+8oDJEttqA3Nzfrvvvuq4Bus802eu+993a4yXxxFpGDEl2VxTjLaH4n2/lxE5FsiVlRZV9myhGpqHDEJdcM32wNPVUsczXQfOI0YXgHUQS5v/zySz3ttNMUnLT5n/70p7pq1arwHlhkYisi+W5xExGv33jFHPoNqjZq+parXkmqGETtiahGl07/yCOPaM+ePRXQ7bffXmfMmNEhvBITkZDwO0elmDb52dK7R7nEIOqYSKodUQjJp59+qieeeKImc0o2b95cnAeHiIlISAQ9WzYbXhtE0J6Il9hJJjFwG51xm4gXNFHPHm5padHJkye3SUor5QLRJiIh4eUPNYiRgnwaRD5BUy9btpyRpBj6Ke0YVsOOYx2Ts88+W4cPH65r166NzogCMREJkVwegtcZtdkotAqZ165ILpHIlk+SD8Vs2HGrqPbRRx9p165dFdDevXvriy++GI0hBWIiEiFuM2qT3/JeuimFNIh8PJJcc23cMlvz9SCK2bDj6Im89957+v3vf1+TIzgXXXSRbtiwITqD8sBEJGKyDal6CTYW0iByxUbKy9sKl5fCzm5i5zdeE0bDjjom4kZTU5Nef/31WllZqYB+97vf1ddffz1aozxgIhIx+Y6YZBoizrdB5Hpm+rd/oY3Ob7wmzIYdp6pp6bzzzjtaV1engJ533nlRm5MTE5GIyXfExK1Ycj4NItczM337F9LoCo3XxLFhF5tNmzbpDTfcoOvXr2/dF9dZwSYiETNjRn7eSBDufbaYSJDf/vkIoZGdDRs26P7776/jx4+P3XBwNhGJtLJZZ+GMM+CCC5yqYKlUVUFlZdt93brB+PHt75FevWzmzOzHwak+1qdP2/PKy+GII5wKZ273ciP9Gb/5jfu5PXt6u6exlblz57J06VLGjh3LgAEDWL58edQmecNNXeK4laonkqTQ6fG5YgluGaLJYKnXNXNz2e61gDO4D/9adyY7L7zwgu62224KaHV1td55552xKDGAdWdKm1xxhyCyVHN1oYKK68RxxCRufP311zp8+HAFZyh44MCBumLFikhtyiYi1p0pAdwKFyf3B1HYONc98n1GbW37fWPHwsaNbfdt3Ji7eHRnY/vtt2fq1KnMmTOHXr168dJLL/HGG29EbZYrJiIlQKYGmbrf7XgQz/B6PJWqqsxxnVxiaLRl2LBhLF26lNtvv53TTjutdf/mzZsjtKo9JiIlwPjxTsA1ldQAbKbj+eAWzM1lQyZqauDBBzMvJ5FLDI329OrVi1GjRrV+Xrx4MbvvvjuPPvpohFal4dbPiePWWWMiqt5KEqZWN0vPhM0UA8k3uJltMp6XYWmLifjn0ksvbY2VnHbaaUVbL5g4B1aBcpxq7/+b69ygRaQURgoKtXHGjODXdXELribn14T5PoZDS0uL/vGPf9RtttlGAd1ll1306aefDv25cReRy4GHii0ipfCt6NfGoN8x25BukJjQ5ObDDz/UI488stUrOe+889pkvgZNbEUE2A2YBxxTbBGJ4yzPdIKwMcgGWYx/s1IQ97jQ3NysN998s1ZVVek222wTaqX5OIvI/wCHAke7iQghrTsTt3oTmSi2jYUsBZGe9OZXsEpB3OPGkiVLdPbs2a2fW1patKGhIdBnxFJEgJOAiYnfXUUkdesInkg+DS2OU+fd7A/KgygFcY87EydO1H322UffeuutwO4ZVxH5L2AVsAL4HNgIzMh2TanHRPJ9pp/p+WHMxs1236AEzzwRf2zZskX79++v4KwXfNVVVwVSKDqWItLGiAg8EdXiB/AKaSD52ugmPBdemP0+uTyAXIIWlAdhMRH/bNy4US+//HIVEQX04IMP1vr6el/3NBGJCcVw1bMNw2ZrmIXOz/F6PB9sdCYYXn75Ze3bt68CWlVVpTfddJM2NzcXdK/Yi4jXrdRFpBiueqF1S/x6GuZBxJP169freeedp4AecsghBRc9MhGJCUE3tEzf2PnMtk0miaVmuqYut5lvzMM8iPjyzDPP6JIlS1o/b9iwIa+V+UxEYkRQDS1b7MNrlfeamvjWRzXCo6WlRX/+85/r4MGD9fPPP/d0jYlIB8TNM0imurulvKcKQK4K7+mYp9ExWLlypdbU1Ghtba2uW7fO0zXZRMRm8ZYobtPnt2zZ+jO9HGOS8nKndOJXX+V37zPOgBUroKXF+Zlppq4XcpV6NMKld+/eLFmyhNmzZ9O9e3ff9zMRKVG8TJ9XbS8k3brBtGmOAEQxNX/mTBg5Ej7+2LHv44+dzyYkxWXnnXfm0EMPDeReJiIlitf6HqpOsWYR5+ekSY6AzJwJGza0P99LbRE/WHWzjoeJSIzJ5vafccbWau4iThclE336tO+CJL2BNWvanltTs1VkwsKqm3U8TERiihe3PzVGMW1a9upnqWTyBgC23TY/ASkktmHVzTogbhHXOG6daXQmzBT5IDJni7HsphEfyDI6I87x0qB///66YMGCqM0oCmVlThNLR8TxPPywww7tuzLgdGe+/NLbPfr2dbyjdJLdp2zMnOl4QytXOh7I+PHhdqEM/4jIQlXtn+mYdWdiStzdfj+xjWxDxTb8W3qYiMSUXBXe/eCWH+K2PxNhiJwN/5YmJiIxJX30JXV4Nkmh39pBCEAYImfDvyWKW7AkjltnCqzmwk+AMqjgZtBp8FbVLL5ggdWOh5/AJsQzuOn3nYzwsMBqB8Rv0lZQ82CCJMw4kBEekYmIiPQWkZdEZJmILBWRS6OypRSJ++hNIXiJAxnxI0pPpBkYrar7AocDF4nIfhHaU1J01G/tOHpIRnYiExFV/UxV3078/g2wDNg1KntKDfvWNuJCLAKrItIXeBU4QFXXpx0bibOAFbW1tYd+nCnyZhhGqMQ6sCoi2wKPAaPSBQRAVSepan9V7d+rV6/iG2gYRlYiFRERqcQRkJmqOjtKWwzDKIwoR2cEmAwsU9XborKj1LG5JkbUROmJHAmcBRwjIu8mthMitKfksLkmRhyIRWDVK5ax2hbL8DSKRawDq0bhWKlBIw6YiJQwHTFr1Sg9TERKmI6atWqUFiYiJYxlrRpxoCJqAwx/nHGGiYYRLeaJGIbhCxMRwzB8YSISQywL1SglLCYSM5JZqMmCxcksVLDYhxFPzBOJGVbx3Cg1TERihmWhGqWGiUjMsCxUo9QwEYkZloVqlBomIjHDslCNUsNGZ2KIZaEapUTU5RGPE5HlIvKhiPwuSlsMwyiMKMsjlgP3AMcD+wGn27ozhlF6ROmJfB/4UFX/oaqNwMPA0AjtMQyjAKIUkV2BT1I+ryLD4lUiMlJEFojIgtWrVxfNOMMwvBGliEiGfe0Kvtq6M4YRb6IcnVkF9E75vBvwabYLFi5c+KWIFHMJvB2AL4v4vKjpTO9r75offdwORFbtXUQqgPeBQcA/gb8B/66qSyMxKAMissCtwnVHpDO9r71rcETmiahqs4hcDDwHlAMPxklADMPwRqTJZqr6DPBMlDYYhuEPS3vPzqSoDSgynel97V0DoqRWwDMMI36YJ2IYhi9MRAzD8IWJiAudaXKgiKwQkXoReVdEOtyK6SLyoIh8ISJLUvb1FJEXROSDxM/vRGljULi86zUi8s/E/++7InJCkM80EclAJ50cOFBVD+qguRNTgePS9v0OmKeqewHzEp87AlNp/64Atyf+fw9KjIoGholIZmxyYAdCVV8FvkrbPRSYlvh9GjCsqEaFhMu7hoqJSGY8TQ7sQCjwvIgsFJGRURtTJHZS1c8AEj93jNiesLlYRBYnujuBdt1MRDLjaXJgB+JIVT0Ep/t2kYj8KGqDjED5I7AncBDwGTAhyJubiGQm78mBpYyqfpr4+QUwB6c719H5l4jsApD4+UXE9oSGqv5LVbeoagtwPwH//5qIZOZvwF4isruIVAGnAU9GbFMoiMg2IrJd8nfgWGBJ9qs6BE8CwxO/DweeiNCWUEmKZYJTCPj/1wo1Z6CTTQ7cCZgjIuD8PTykqs9Ga1KwiMgs4GhgBxFZBYwDbgQeFZFzgZXAqdFZGBwu73q0iByE0yVfAZwf6DMt7d0wDD9Yd8YwDF+YiBiG4QsTEcMwfGEiYhiGL0xEDMPwhYmIYRi+MBExQkFEXhKRwYnfrxeRu6K2yQgHSzYzwmIc8AcR2RE4GBgSsT1GSFiymREaIvIKsC1wtKp+IyJ7AGOBHqr6i2itM4LCujNGKIhIHbALsFlVvwFI1Gc5N1rLjKAxETECJzHhayZO4Z9vReSnEZtkhIiJiBEoItINmA2MVtVlwHXANZEaZYSKxUSMoiEiNcB4YDDwgKr+V8QmGQFgImIYhi+sO2MYhi9MRAzD8IWJiGEYvjARMQzDFyYihmH4wkTEMAxfmIgYhuELExHDMHxhImIYhi/+P4o1tgX9CDBSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import sklearn\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "scikit_log_reg = sklearn.linear_model.LogisticRegression();\n",
    "scikit_log_reg.fit(X,y)\n",
    "\n",
    "#Score is Mean Accuracy\n",
    "scikit_score = scikit_log_reg.score(X,y)\n",
    "print('Scikit score: ', scikit_score)\n",
    "\n",
    "# Print accuracy\n",
    "y_predictions = scikit_log_reg.predict(X)\n",
    "print ('预测的准确度是: %d ' % float((np.dot(y, y_predictions) \n",
    "        + np.dot(1 - y,1 - y_predictions)) / float(y.size) * 100) +    '% ' )\n",
    "\n",
    "#plot_decision_boundary(lambda x: clf.predict(x), X, Y)\n",
    "# Plot decision boundary\n",
    "x1 = np.array([X[:,0].min()-1, X[:,0].max()+1])\n",
    "x2 = - w.item(0) / w.item(2) + x1 * (- w.item(1) / w.item(2))\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(4,4))\n",
    "ax.scatter(X[:n_pts,0], X[:n_pts,1], color='lightcoral',\n",
    "          label='$Y = 0$')\n",
    "ax.scatter(X[n_pts:,0], X[n_pts:,1], color='blue',\n",
    "          label='$Y = 1$')\n",
    "ax.set_title('Sample Dataset')\n",
    "ax.set_xlabel('$x_1$')\n",
    "ax.set_ylabel('$x_2$')\n",
    "\n",
    "ax.plot(x1, x2, color='k', ls='--', lw=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
